---
doc_version: "1.0.0"
last_updated: "2026-07-14T15:00:57.718Z"
title: "How to Add a Website Source"
description: "Step-by-step guide to adding a website as a knowledge source in ChatbotIQ, from entering the URL to managing crawled pages."
keywords: [add source, website, knowledge base, crawl, index, ChatbotIQ]
---

This guide walks you through adding a website to your knowledge base so your bot can answer questions about its content.

> Looking for other source types? See [Source Types](/reference/source-types/) for PDFs, Q&A pairs, spreadsheets, and more.

---

## Add a new source

1. Go to **Knowledge Base** in the sidebar.
2. Click **"+ Add Source"**.
3. Choose **Website** as the source type.
4. Enter the base URL of the site you want to crawl (e.g., `https://docs.example.com`).
5. ChatbotIQ automatically discovers the pages on your site. This can take a few seconds to a few minutes depending on the site size.
6. Review the discovered pages. You can select or deselect individual pages before crawling.
7. Click **Start Crawling** (the button shows the page count, e.g. "Start Crawling 47 Pages") to begin downloading and indexing the content.

> **First time?** Start with a smaller site (under 100 pages) to see how the crawler handles your content. You can always add more sources later.

---

## Understand source statuses

Each source in your knowledge base shows its current state:

| Status | What it means |
|--------|---------------|
| **Discovering** | ChatbotIQ is scanning the site for pages (reading the sitemap or following links). |
| **Crawling** | Pages are being downloaded. You'll see progress like "12/47 pages crawled". |
| **Indexing** | Downloaded content is being converted into embeddings so your bot can search it. |
| **Ready** | Everything is indexed. Your bot can now use this source to answer questions. |
| **Failed** | Something went wrong. You can retry failed pages without re-crawling everything. |

---

## Manage your sources

From the Knowledge Base page, you can:

- **Refresh** - re-crawl an existing source to pick up content changes. ChatbotIQ shows you which pages changed, which stayed the same, and which failed.
- **Cancel** - stop a running crawl or discovery at any time.
- **Retry failed pages** - if some pages failed during a crawl, retry just those pages without re-crawling the entire site.
- **View report** - after a crawl completes, see a detailed breakdown: every page with its status, the number of content chunks created, and page-by-page details.
- **Delete** - remove a source and all its indexed content. Your page count decreases automatically.

---

## Tips for better results

- **Enable Reader Mode** if your pages include navigation bars, sidebars, or footer content that shouldn't be in your knowledge base. See [Configure Crawling for Your Site](/how-to/configure-crawling-for-your-site/).
- **Check the crawl report** after your first crawl. Look for pages with very little content - they might need different crawl settings.
- **Use the Playground debug panel** to test how well your bot retrieves content from this source.

---

## Related

- [Configure Crawling for Your Site](/how-to/configure-crawling-for-your-site/) - optimize crawling for WordPress, Confluence, SPAs, and more
- [Source Types](/reference/source-types/) - all supported source formats
- [Keep Content Up to Date](/how-to/refresh-and-maintain-sources/) - when and how to refresh your sources
- [How Web Crawling Works](/explanation/how-crawling-works/) - understand what happens behind the scenes

## Sitemap

Full documentation index: [sitemap.md](/sitemap.md) and [llms.txt](/llms.txt).