How Knowledge Base Retrieval Integrates with Voice Pipeline Context in Dograh

Dograh’s Pipecat voice pipeline automatically injects a retrieval tool into the per-turn LLM context when workflow nodes reference Knowledge Base documents, binding vector search results directly to the conversational state.

Dograh’s voice infrastructure leverages the Pipecat engine to manage real-time, turn-based conversations with large language models. When a workflow node includes document_uuids from the Knowledge Base, the system dynamically registers the retrieve_from_knowledge_base function as an available tool for that specific turn. This architecture ensures that knowledge base retrieval executes as a native extension of the voice pipeline context, allowing the LLM to ground its responses in relevant, similarity-ranked document excerpts.

The Four-Stage Integration Flow

The integration follows a strict pipeline: definition, registration, composition, and execution. Each stage couples the retrieval capability tightly to the active LLMContext, ensuring traceability and stateful conversation management.

Stage 1: Tool Definition in the Knowledge Base Service

The retrieval specification lives in api/services/workflow/tools/knowledge_base.py. This file implements the retrieve_from_knowledge_base async function and constructs the OpenAI-style function schema that tells the LLM how to invoke document search.

The function signature accepts critical context parameters:

• query: The search string extracted from user intent
• document_uuids: A filtered list of specific documents to search (optional)
• top_k: The number of chunks to return (default 5)
• parent_context: The active LLMContext object for tracing and organization scoping

Stage 2: Dynamic Registration in the Pipecat Engine

Inside api/services/workflow/pipecat_engine.py, the _register_knowledge_base_function method handles conditional tool registration. When processing a workflow node that contains document_uuids, the engine executes:

if node.document_uuids:
    # Attach the KB retrieval function to the LLM for this call turn

    await self._register_knowledge_base_function(node.document_uuids)

This registration binds the async retrieval coroutine to the LLM client instance, making it available for function calling during that specific conversational turn.

Stage 3: Context Composition

The api/services/workflow/pipecat_engine_context_composer.py assembles the complete toolset before each LLM interaction. It calls get_knowledge_base_tool and adds the resulting definition via engine.llm.register_function. This composition step ensures that knowledge base retrieval appears alongside other built-in tools in the PipecatEngine’s execution context.

Stage 4: Runtime Execution and Context Propagation

When the LLM decides to answer a question using retrieved knowledge, it emits a function call that the Pipecat engine intercepts. The execution flow proceeds as follows:

1. Function Invocation: The engine invokes retrieve_from_knowledge_base with the parent_context attached
2. Vector Search: The function calls KnowledgeBaseClient.search_documents (defined in api/db/knowledge_base_client.py) to perform a vector similarity search against the stored chunks
3. Context Enrichment: The function extracts parent_context.organization_id to propagate trace attributes (specifically gen_ai.operation.name = knowledge_base_retrieval)
4. Result Injection: The top-k text excerpts return to the LLM as function results, effectively becoming part of the next prompt’s context

Code Implementation Examples

Registering the Retrieval Tool for Document-Enabled Nodes

When the voice pipeline processes a node configured with knowledge base documents, the engine conditionally attaches the retrieval capability:

# Inside PipecatEngine when a node is being processed

if node.document_uuids:
    # Attach the KB retrieval function to the LLM for this call turn

    await self._register_knowledge_base_function(node.document_uuids)

The Retrieval Coroutine with Parent Context Access

The actual implementation leverages the parent_context to maintain observability and multi-tenancy:

async def retrieve_from_knowledge_base(
    query: str,
    document_uuids: List[str] | None = None,
    top_k: int = 5,
    parent_context: Optional[LLMContext] = None,
) -> dict:
    # Obtain the organization ID from the parent context (used for tracing)

    org_id = parent_context.organization_id if parent_context else None

    # Perform a vector similarity search against the chosen documents

    results = await KnowledgeBaseClient.search_documents(
        organization_id=org_id,
        query=query,
        document_uuids=document_uuids,
        top_k=top_k,
    )

    # Return the excerpts that the LLM will see in its next prompt

    return {"results": [r.text for r in results]}

LLM Function Call Flow

The model receives the tool definition and generates calls like:

{
  "role": "assistant",
  "content": null,
  "function_call": {
    "name": "retrieve_from_knowledge_base",
    "arguments": {
      "query": "What are our refund policies?",
      "document_uuids": ["d5f2c8a1-…"],
      "top_k": 3
    }
  }
}

The Pipecat engine executes this call, and the returned snippets become visible context for response generation:

User: I’d like to know about refunds.
[Knowledge‑Base Retrieval] → “Our refund policy is 30 days …”
Assistant: Sure! You can request a refund within 30 days …

Tracing and Observability Benefits

Because the retrieval function receives the parent LLM context (parent_context), the system achieves several observability goals:

• Operation Tracing: Each retrieval is tagged with gen_ai.operation.name = knowledge_base_retrieval, allowing distributed tracing systems to isolate search latency and failure rates
• Metadata Attachment: Retrieved text attaches to call-level metadata, enabling downstream processors (TTS, logging, analytics) to see exactly which knowledge base snippets influenced the assistant’s answer
• Correlation: The organization_id extracted from context ensures multi-tenant isolation, preventing cross-organization document leakage

Summary

• Knowledge base retrieval in Dograh operates as a dynamic tool registered per-turn within the Pipecat voice pipeline
• The retrieve_from_knowledge_base function in api/services/workflow/tools/knowledge_base.py defines both the schema and vector search execution logic
• Registration occurs conditionally in api/services/workflow/pipecat_engine.py via _register_knowledge_base_function when nodes contain document_uuids
• The parent_context parameter links every retrieval operation to the active conversation state, enabling traceable, organization-scoped document search
• Retrieved excerpts flow back into the LLM prompt stream, grounding responses in factual document content

Frequently Asked Questions

How does the retrieval function access conversation metadata?

The retrieve_from_knowledge_base function accepts a parent_context parameter of type LLMContext. This object contains the active organization_id and other call-level attributes, allowing the retrieval to propagate trace information and enforce multi-tenant isolation while maintaining correlation with the specific voice call.

What triggers knowledge base retrieval during a voice call?

Retrieval triggers when the Pipecat engine processes a workflow node that includes document_uuids. The engine checks for these UUIDs in api/services/workflow/pipecat_engine.py and calls _register_knowledge_base_function to make the tool available. The actual execution occurs when the LLM decides to invoke the function based on user intent.

Which component performs the vector similarity search?

The KnowledgeBaseClient.search_documents method, implemented in api/db/knowledge_base_client.py, executes the vector similarity search. The retrieval tool defined in api/services/workflow/tools/knowledge_base.py wraps this client call, passing the query, document_uuids, and organization_id parameters to filter and rank document chunks.

How are retrieved documents injected back into the conversation?

Upon completion of retrieve_from_knowledge_base, the function returns a dictionary containing the top-k text excerpts. The Pipecat engine injects these results into the LLM’s message history as function return values, making the retrieved content available as context for the assistant’s next response generation step.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}