Getting Started with Knowledge

Overview

Knowledge collections allow you to build a searchable knowledge base from your documents, websites, and other content sources. Your chatbot can then retrieve relevant information to provide accurate, context-aware responses.

Core Concepts

Collections

A collection is a container for related knowledge. You might have separate collections for different topics, departments, or use cases (e.g., "Product Documentation", "HR Policies", "Customer FAQs").

Documents

A document represents a single piece of content from a source—a webpage, uploaded file, or manual entry. Each document is automatically processed and broken down into smaller pieces.

Semantic Search

The system uses semantic search to understand the meaning of questions, not just keywords. This means it can find relevant information even when exact words don't match—for example, finding "refund procedure" when a user asks "how do I get my money back?"

Creating Your First Collection

1. Navigate to Knowledge
   From your dashboard, go to the Knowledge section in the sidebar.
2. Create Collection
   Click "New Collection" and give it a descriptive name (e.g., "Product Documentation").
3. Add a Description (optional)
   Describe what content this collection contains. This helps when you have multiple collections.
4. Save the Collection
   Your empty collection is now ready to receive content.

Adding Content to Collections

There are four ways to add content to a knowledge collection:

1. Upload Files

Upload documents directly to your collection. Supported formats include:

• Text files (.txt, .md)
• HTML files (.html)
• PDF files (.pdf)
• Other document formats processed by the system

The system automatically extracts text content and converts it to a searchable format.

2. Scrape Websites

Point to a website and let the system crawl and index its content:

• Enter a base URL (e.g., https://docs.yoursite.com)
• Set a maximum page limit to control crawling scope
• The crawler follows internal links and indexes all discovered pages
• Content is automatically converted from HTML to markdown

3. Add Specific URLs

Index specific pages without crawling entire sites:

• Provide a list of URLs to scrape
• Each URL is fetched and processed independently
• Perfect for curating content from multiple sources

4. Manual Entry

Type or paste content directly:

• Create documents manually in the UI
• Paste text from other sources
• Ideal for quick additions or proprietary content

Learn more about each content source type in the Document Sources guide.

How Content is Processed

When you add content to a collection, it's automatically processed to make it searchable and optimized for AI retrieval. Here's what happens:

Using Knowledge in Your Flows

Once your collection has processed content, you can use it in conversational flows. We recommend starting with the Knowledge Agent node for the simplest and most powerful experience.

Knowledge Agent (Recommended)

The Knowledge Agent node provides an all-in-one solution for answering questions using your knowledge base:

• Automatically searches your collections to find relevant information
• Generates accurate, well-cited answers based on retrieved content
• Handles follow-up questions with conversation context
• Cites sources so users can verify information
• No additional nodes needed—it handles everything internally

Advanced: RAG Retriever Nodes

For more control over the retrieval and response generation process, you can use RAG Retriever nodes:

• RAG Retriever: Basic search and retrieval. Returns chunks that you can use with Content Generator nodes for custom responses.
• Agentic RAG Retriever: Advanced retrieval with automatic query reformulation, multi-hop reasoning, and result quality assessment.

These nodes give you fine-grained control but require connecting additional nodes (like Content Generator) to create complete responses.