Knowledge Base

Build your chatbot's knowledge base by uploading documents or configuring automated document sources. Documents are processed into searchable chunks with vector embeddings, enabling the chatbot to provide accurate, context-aware responses.

Documents

Documents are individual files or web pages that make up your knowledge base. Each document is split into chunks, enriched with metadata, and vectorized for semantic search.

Supported File Types

• Documents: PDF, DOCX, XLSX, PPTX
• Web Content: HTML, Markdown
• Data: CSV, Plain Text

Uploading Documents

Uploaded documents go through the following processing stages:

1. Loading — File content is extracted
2. Chunking — Content is split into searchable segments
3. Enrichment — Title, description, and locale are generated using AI
4. Contextual Processing — Each chunk receives surrounding context for better retrieval
5. Vectorization — Embeddings are generated for semantic search

Managing Documents

The documents index provides search and filtering:

• Search by title or source URL
• Filter by vectorization status (vectorized or pending)
• Sort by name, date, or file size
• Bulk delete multiple documents at once

Each document displays its processing status, including the number of chunks created and how many have been vectorized.

Workspace Access

Documents can be shared across workspaces. For manually uploaded documents (not from a source), admins can manage workspace access from the document detail page.

Document Sources

Document sources automate the ingestion of documents from external systems. Instead of uploading files manually, configure a source to crawl and import content automatically.

Web Source

Crawl websites to import their content as documents.

Web Source Settings

Crawling Authenticated Websites

If the website you want to crawl is behind a login or requires authentication, you can use the Header setting to pass custom HTTP headers with each request. Headers are semicolon-separated.

Option 1: Basic Authentication

If the website uses HTTP Basic Authentication, add an Authorization header with a Base64-encoded username:password value:

Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

To generate the Base64 value, encode username:password (e.g., echo -n 'username:password' | base64).

Option 2: Re-using a Session Cookie

If you can log into the website manually, you can copy the session cookie from your browser and pass it as a header:

Cookie: session_id=abc123def456

Option 3: API Key or Bearer Token

If the website supports token-based access, use the appropriate header:

Authorization: Bearer your_access_token

or

X-Api-Key: your_api_key

Jira Source

Import Jira issues as documents. Each issue is converted to a structured document containing its summary, description, comments, status, and other metadata.

Note: Jira integration requires the Jira feature to be enabled.

Jira Source Settings

Google Drive Source

Import files from Google Drive into your knowledge base. Supports Google Docs, Sheets, and Slides (automatically converted to DOCX, XLSX, and PPTX), along with all other supported file types.

Note: Google Drive integration requires a Google Drive connector with OAuth authentication.

Google Drive Source Settings

SharePoint Source

Import files from Microsoft SharePoint into your knowledge base. Browse SharePoint sites, drives, and folders to select files for ingestion.

Note: SharePoint integration requires a Microsoft SharePoint connector with OAuth authentication.

SharePoint Source Settings

Processing Sources

After creating a source, click Process to start the crawl. You can monitor progress from the Crawler Runs page, which shows:

• Status — Pending, running, completed, or failed
• Progress — Percentage of completion
• Duration — How long the crawl took
• Error Message — Details if the crawl failed

Periodic Crawling

When Periodic Crawl is enabled on a source, the system automatically re-crawls daily. For Jira sources, only issues updated since the last run are fetched, making subsequent crawls faster.

Workspace Access

Document sources can be assigned to specific workspaces. All documents imported from a source are automatically available to the same workspaces.

Translation

When the translation feature is enabled, documents can be automatically translated to a configured base language. This is useful when your knowledge base contains documents in multiple languages but you want consistent retrieval.

• Enable Translate on the document source
• Set the base translation language in Admin → Settings
• Translated content is used alongside original content for search
• Documents already in the target language are left unchanged