# Best Practices for Optimizing Knowledge Base Retrieval Performance in AstrBot

> Optimize AstrBot knowledge base retrieval performance with our four-stage hybrid pipeline. Discover tunable parameters and advanced techniques for efficient information access.

- Repository: [AstrBot AI/AstrBot](https://github.com/AstrBotDevs/AstrBot)
- Tags: best-practices
- Published: 2026-03-12

---

**AstrBot achieves optimal knowledge base retrieval performance through a four-stage hybrid pipeline that combines dense vector search, BM25 sparse retrieval, reciprocal rank fusion, and optional LLM reranking, with tunable parameters for chunk sizing, top-k thresholds, and index configuration.**

Optimizing knowledge base retrieval performance in AstrBot requires understanding its hybrid architecture that leverages both dense embeddings and sparse keyword matching. The system stores each knowledge base as chunked documents indexed in FAISS vectors and BM25 term frequencies, coordinated by the `RetrievalManager` in [`astrbot/core/knowledge_base/retrieval/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/manager.py). By tuning chunking strategies, retrieval parameters, and index types, you can balance retrieval accuracy against latency for your specific use case.

## Understanding AstrBot's Four-Stage Retrieval Pipeline

The retrieval process in [`astrbot/core/knowledge_base/retrieval/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/manager.py) executes four distinct stages, each configurable via parameters stored in the knowledge base models at [`astrbot/core/knowledge_base/models.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/models.py):

1. **Dense Retrieval** (`_dense_retrieve`): Queries the per-KB FAISS vector database using cosine similarity. Controlled by `top_k_dense` (default 50).
2. **Sparse Retrieval** ([`astrbot/core/knowledge_base/retrieval/sparse_retriever.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/sparse_retriever.py)): Executes BM25 keyword search on raw chunk text. Controlled by `top_k_sparse` (default 50).
3. **Rank Fusion** ([`astrbot/core/knowledge_base/retrieval/rank_fusion.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/rank_fusion.py)): Merges dense and sparse results using Reciprocal Rank Fusion (RRF). Controlled by `top_k_fusion` (default 20).
4. **Reranking** (`_rerank`): Optional LLM-based reranking if `rerank_provider_id` is configured. Returns `top_m_final` chunks (default 5).

## Optimizing Chunk Size and Overlap

The `RecursiveCharacterChunker` in [`astrbot/core/knowledge_base/chunking/recursive.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/chunking/recursive.py) determines how documents are segmented before indexing. Chunk sizing directly impacts vector database size and retrieval granularity.

**Best practices for chunk configuration:**

- **Long-form prose**: Use `chunk_size=512` with `chunk_overlap=50` to maintain context across boundaries while keeping vector counts manageable.
- **Short FAQ-style content**: Reduce `chunk_size` to 256-300 to minimize vector count and improve exact match retrieval.
- **Code documentation**: Use larger chunks (768-1024) with minimal overlap (20-30) to preserve function context.

Configure chunking when creating a knowledge base via `kb_manager.create_kb` in [`astrbot/core/knowledge_base/kb_mgr.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/kb_mgr.py):

```python
kb_helper = await kb_manager.create_kb(
    kb_name="TechnicalDocs",
    embedding_provider_id="openai-embeddings",
    chunk_size=256,
    chunk_overlap=30,
)

```

## Tuning Top-K Parameters for Latency vs. Recall

The `top_k` parameters control the candidate pool size at each retrieval stage. Adjust these in `kb_manager.update_kb` to optimize for your latency budget:

- **`top_k_dense` and `top_k_sparse`**: Increase (e.g., to 100) for higher recall on ambiguous queries; decrease (e.g., to 20) for strict latency requirements.
- **`top_k_fusion`**: Controls how many fused candidates proceed to reranking. Values between 10-30 typically provide the best precision-latency balance.
- **`top_m_final`**: The actual chunks returned to users. Keep this ≤ 5 unless the downstream LLM context window supports large inputs.

Example configuration update:

```python
await kb_manager.update_kb(
    kb_id=kb_helper.kb.kb_id,
    kb_name="TechnicalDocs",
    top_k_dense=30,
    top_k_sparse=30,
    top_k_fusion=15,
    top_m_final=3,
    rerank_provider_id=None,  # Disable reranker for speed

)

```

## Configuring BM25 Stop-Words for Sparse Retrieval

The sparse retriever in [`astrbot/core/knowledge_base/retrieval/sparse_retriever.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/sparse_retriever.py) loads stop-words from [`hit_stopwords.txt`](https://github.com/AstrBotDevs/AstrBot/blob/main/hit_stopwords.txt) to exclude high-frequency, low-information terms from BM25 scoring.

**Optimization strategies:**

- Add domain-specific noise terms (e.g., "example", "demo", "test" in documentation KBs) to reduce false positives.
- Keep the stop-word list under 200 entries to maintain scoring performance.
- Modify the file at runtime, then restart the AstrBot process to reload the module.

Example stop-word extension:

```python
from pathlib import Path

STOPWORDS_PATH = Path("astrbot/core/knowledge_base/retrieval/hit_stopwords.txt")
custom_stopwords = {"example", "demo", "test", "placeholder"}

with STOPWORDS_PATH.open("a", encoding="utf-8") as f:
    for word in custom_stopwords:
        f.write(f"{word}\n")

```

## Selecting Vector Index Types: Flat vs. IVF

AstrBot uses `FaissVecDB` in [`astrbot/core/db/vec_db/faiss_impl.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/db/vec_db/faiss_impl.py) for vector storage, defaulting to a flat index that loads entirely into RAM. For knowledge bases exceeding 100,000 chunks, consider migrating to an **IVF (Inverted File)** index to reduce memory footprint and query latency.

**Index selection guidelines:**

- **Flat Index** (`IndexFlatIP` or `IndexFlatL2`): Best for KBs under 50k chunks; provides exact nearest neighbor search.
- **IVF Index** (`IndexIVFFlat`): Partition vectors into `nlist` clusters; queries only the nearest clusters. Use `nlist=4096` for 100k+ chunks, adjusting `nprobe` (clusters to search) for recall/latency trade-offs.

Note: Switching index types requires rebuilding the FAISS index from stored embeddings in [`astrbot/core/db/vec_db/faiss_impl.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/db/vec_db/faiss_impl.py).

## Managing Embedding Providers and Rerankers

Provider configuration in [`astrbot/core/provider/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/manager.py) significantly impacts retrieval quality and speed.

**Embedding provider selection:**

- **OpenAI `text-embedding-3-small`**: 1536 dimensions, lower latency, suitable for general-purpose KBs.
- **OpenAI `text-embedding-3-large`**: 3072 dimensions, higher quality, increased latency and storage cost.
- **Local models**: Use for air-gapped environments, but verify batch processing support in [`astrbot/core/provider/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/manager.py).

**Reranker optimization:**

- Enable reranking only when `rerank_provider_id` supports the `rerank` method (e.g., Cohere or OpenAI rerankers).
- Reranking adds 100-500ms latency per query due to additional LLM calls.
- **Best practice**: Disable reranking (`rerank_provider_id=None`) for latency-sensitive applications; enable only when precision is critical and `top_m_final` ≤ 5.

## Monitoring Retrieval Performance

AstrBot logs detailed timing metrics via `logger.debug` in `RetrievalManager.retrieve` within [`astrbot/core/knowledge_base/retrieval/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/manager.py).

**Key metrics to monitor:**

```

Dense retrieval across 2 bases took 0.12s and returned 84 results.
Sparse retrieval across 2 bases took 0.04s and returned 100 results.
Rank fusion took 0.01s and returned 20 results.

```

**Optimization workflow:**

1. Enable debug logging to capture stage-specific timings.
2. Identify the bottleneck (typically dense retrieval for large KBs or reranking when enabled).
3. Adjust the corresponding `top_k` parameters or index type.
4. Target a combined retrieval time under 300ms for interactive chat applications.

## Summary

- **AstrBot's retrieval pipeline** combines dense FAISS search, BM25 sparse retrieval, RRF fusion, and optional LLM reranking through [`astrbot/core/knowledge_base/retrieval/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/manager.py).
- **Chunk sizing** directly impacts vector count and retrieval granularity; use 256-512 tokens depending on content type via [`astrbot/core/knowledge_base/chunking/recursive.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/chunking/recursive.py).
- **Top-K tuning** balances latency and recall: reduce `top_k_dense`, `top_k_sparse`, and `top_k_fusion` for speed, or increase them for comprehensive recall.
- **BM25 optimization** requires maintaining a concise stop-word list in [`hit_stopwords.txt`](https://github.com/AstrBotDevs/AstrBot/blob/main/hit_stopwords.txt) loaded by [`astrbot/core/knowledge_base/retrieval/sparse_retriever.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/sparse_retriever.py).
- **Index selection** matters for scale: use flat FAISS indices for small KBs and consider IVF indices for 100k+ chunks in [`astrbot/core/db/vec_db/faiss_impl.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/db/vec_db/faiss_impl.py).
- **Reranker usage** adds significant latency; disable `rerank_provider_id` unless precision is critical.

## Frequently Asked Questions

### How do I reduce latency for real-time chat applications using AstrBot's knowledge base?

To reduce latency, decrease the candidate pool sizes by setting `top_k_dense` and `top_k_sparse` to 20-30 instead of the default 50, and reduce `top_k_fusion` to 10-15. Disable the reranker by setting `rerank_provider_id=None` in [`astrbot/core/knowledge_base/models.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/models.py), as reranking adds 100-500ms per query. Finally, ensure your FAISS index fits in RAM to avoid disk swapping.

### What is the optimal chunk size for technical documentation in AstrBot?

For technical documentation, use a `chunk_size` of 512 tokens with `chunk_overlap` of 50 tokens via the `RecursiveCharacterChunker` in [`astrbot/core/knowledge_base/chunking/recursive.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/chunking/recursive.py). This preserves code context and explanatory paragraphs while keeping vector database size manageable. For FAQ-style content with short answers, reduce `chunk_size` to 256-300 tokens to minimize vector count and improve exact-match retrieval.

### When should I use an IVF index instead of the default flat FAISS index in AstrBot?

Switch to an IVF (Inverted File) index in [`astrbot/core/db/vec_db/faiss_impl.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/db/vec_db/faiss_impl.py) when your knowledge base exceeds 100,000 chunks or when memory constraints prevent loading the entire flat index into RAM. IVF indices partition vectors into clusters (e.g., `nlist=4096`), querying only the nearest clusters to reduce both memory footprint and search latency. Note that IVF requires rebuilding the index from stored embeddings and may require tuning `nprobe` (clusters to search) to balance recall versus speed.

### How does the reciprocal rank fusion (RRF) work in AstrBot's retrieval pipeline?

Reciprocal Rank Fusion in [`astrbot/core/knowledge_base/retrieval/rank_fusion.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/knowledge_base/retrieval/rank_fusion.py) combines results from dense FAISS retrieval and sparse BM25 retrieval without requiring score calibration between the different modalities. RRF assigns each document a fused score based on the sum of reciprocal ranks (1/(k + rank)) from each retrieval method, where k is a constant (typically 60). This ensures that documents appearing highly ranked in both dense and sparse results surface to the top, while the `top_k_fusion` parameter (default 20) controls how many fused candidates proceed to optional reranking or final output.