How to Build an Agentic RAG System with Query Reasoning and Retrieval Planning

An agentic RAG system extends traditional retrieval-augmented generation by inserting an autonomous reasoning loop that assesses information gaps, plans retrieval steps, executes them, and adapts strategy until sufficient context is gathered.

The davidkimai/context-engineering repository implements this architecture as a three-layer Software 3.0 stack, providing concrete Python classes and protocol shells that transform static RAG pipelines into dynamic, self-correcting research agents.

Understanding the Agentic RAG Architecture

Traditional RAG follows a linear path: query, retrieve, generate. Agentic RAG introduces a cognitive control layer that treats retrieval as a strategic decision-making process rather than a single function call.

The Three-Layer Software 3.0 Stack

The repository structures agentic RAG into three distinct layers, each defined in 00_COURSE/04_retrieval_augmented_generation/02_agentic_rag.md:

Layer	Responsibility	Key Components
Prompt Communication	Conversational templates describing current query, known facts, gaps, and retrieval plans.	AGENT_REASONING_TEMPLATE (basic), STRATEGIC_AGENT_TEMPLATE (intermediate), META_COGNITIVE_AGENT_TEMPLATE (advanced)
Programming Implementation	Python classes that execute assessment, planning, retrieval, and synthesis.	BasicRAGAgent, StrategicRAGAgent, MetaCognitiveRAGAgent
Protocol Orchestration	Structured protocol shells (e.g., /agent.rag.basic{…}) that formalize steps for chaining, monitoring, and auditing.	Protocol definitions throughout the agentic RAG specification

Core Architectural Flow

The CompleteAgenticRAG class orchestrates the full pipeline through six distinct phases:

1. Initial Query Reception – The user query enters the orchestration layer.
2. Reasoning Session Init – The system populates AGENT_REASONING_TEMPLATE with the query and current context.
3. Information Assessment – The LLM returns a structured assessment parsed by assess_information_state, evaluating completeness, gaps, and confidence.
4. Retrieval Planning – plan_next_retrieval crafts a strategy specifying tools, targets, and expected outcomes.
5. Execution Loop – The agent iterates (default max 5 iterations) calling execute_retrieval, updating the session state until information is sufficient or limits are reached.
6. Synthesis – synthesize_response generates the final answer, optionally enriched by a knowledge synthesis protocol.

Implementing Query Reasoning and Retrieval Planning

The transition from basic to agentic RAG centers on two capabilities: reasoning about information state and planning retrieval actions.

Information Assessment and Gap Analysis

The assess_information_state method (defined in the BasicRAGAgent class) parses LLM outputs to determine whether current context satisfies the query. The method evaluates:

• Completeness: Percentage of query aspects covered
• Gaps: Specific missing information or ambiguities
• Confidence: Reliability assessment of current sources

This assessment triggers the planning phase only when gaps are identified, preventing unnecessary retrieval calls for simple queries.

Dynamic Retrieval Planning

The plan_next_retrieval method constructs a structured retrieval strategy by combining:

• Available tools: Retrieved via self.get_available_tools()
• Previous attempts: Accessed through self.memory.get_previous_attempts()
• Current gaps: From the assessment phase

The planning prompt (lines 165-170 in the specification) instructs the LLM to return a JSON plan:

{
  "strategy": "multi-source semantic search",
  "targets": ["paper-XYZ", "API-docs"],
  "tools": ["semantic_search", "vector_db_query"],
  "expected_outcomes": "relevant excerpts + confidence scores"
}

Execution and Adaptation

The execution loop processes the plan through execute_retrieval, which invokes specific tools from the retrieval library. For higher complexity queries, the StrategicRAGAgent adds adaptive execution through should_adapt_strategy, allowing mid-execution re-planning when initial strategies fail.

The MetaCognitiveRAGAgent extends this further with recursive_research_loop, enabling the agent to evaluate its own research quality and upgrade reasoning modules dynamically.

Code Implementation: From Basic to Meta-Cognitive Agents

The repository provides concrete Python implementations for each layer of the agentic stack.

Basic RAG Agent Implementation

The BasicRAGAgent class provides the foundational reasoning loop. Located in 00_COURSE/04_retrieval_augmented_generation/02_agentic_rag.md (lines 107-150), this class implements assess_information_state, plan_next_retrieval, and synthesize_response.

from pathlib import Path
from my_project.agents import BasicRAGAgent, RetrievalTools, ReasoningTemplates

# Load retrieval utilities (vector DB, web search, etc.)

retrieval_tools = RetrievalTools.from_config(Path("config/retrieval.yaml"))

# Load prompt templates

templates = ReasoningTemplates.from_dir(Path("templates/agentic"))

# Instantiate the basic agent

basic_agent = BasicRAGAgent(retrieval_tools, templates)

# Run a query

answer = basic_agent.process_query(
    "What are the main challenges of quantum-aware language models?"
)
print(answer)

Strategic Agent with Multi-Phase Planning

The StrategicRAGAgent (lines 298-346) adds mission analysis and adaptive execution for complex, multi-faceted queries.

from my_project.agents import StrategicRAGAgent, RetrievalTools, ReasoningTemplates, StrategyLibrary

tools = RetrievalTools.from_config("config/retrieval.yaml")
templates = ReasoningTemplates.from_dir("templates/strategic")
strategy_lib = StrategyLibrary.load("strategies/strategic.yaml")

strategic_agent = StrategicRAGAgent(tools, templates, strategy_lib)

response = strategic_agent.process_complex_query(
    "Compare the ethical implications of autonomous weapons versus autonomous medical diagnosis systems."
)
print(response)

Meta-Cognitive Agent with Self-Improvement

The MetaCognitiveRAGAgent (lines 442-466) implements recursive research loops and self-evaluation for autonomous research tasks.

from my_project.agents import MetaCognitiveRAGAgent, RetrievalTools, ReasoningTemplates, MetaEngine

tools = RetrievalTools.from_config("config/retrieval.yaml")
templates = ReasoningTemplates.from_dir("templates/meta")
meta_engine = MetaEngine.from_config("config/meta_engine.yaml")

meta_agent = MetaCognitiveRAGAgent(tools, templates, meta_engine)

research = meta_agent.conduct_research(
    research_question="Assess the long-term societal impacts of AI-augmented education."
)
print(research["research_findings"])
print("Meta-insights:", research["meta_cognitive_insights"])

Complete System Orchestration

The CompleteAgenticRAG class (lines 108-152) orchestrates all three layers, automatically selecting the appropriate agent complexity based on query characteristics.

from my_project.full_system import CompleteAgenticRAG
from my_project.config import SystemConfig

config = SystemConfig.load("config/system.yaml")
rag_system = CompleteAgenticRAG(config)

# The orchestrator automatically picks the appropriate layer

result = rag_system.process_query(
    "Provide a step-by-step plan for deploying a secure, multi-tenant LLM serving platform.",
    complexity_hint="high",
    meta_objectives={"self_improvement": True}
)

print(result)

Configuration and Integration

To wire the agentic RAG system into your infrastructure, configure the retrieval backend and template libraries:

• Retrieval Indexing: Configure vector stores, chunking strategies, and metadata filters in 40_reference/retrieval_indexing.md to support the agents' retrieval executors.
• Cognitive Templates: Reusable prompt fragments for assessment and planning reside in cognitive-tools/cognitive-templates/reasoning.md.
• Program Library: Utility functions for retrieval, memory, and meta-cognitive evaluation are available in cognitive-tools/cognitive-programs/program-library.py.

All agents accept a configuration object bundling retrieval tools, template collections, and optional meta-cognitive engines, enabling plug-and-play deployment across domains (medical, legal, scientific).

Summary

• Agentic RAG transforms static retrieval into an autonomous reasoning loop where the system assesses information gaps, plans retrieval strategies, executes them, and adapts until sufficient context is gathered.
• The three-layer architecture (Prompt Communication, Programming Implementation, Protocol Orchestration) provides separation of concerns between conversational templates, Python execution logic, and structured audit shells.
• Three agent classes—BasicRAGAgent, StrategicRAGAgent, and MetaCognitiveRAGAgent—offer increasing sophistication, from simple gap assessment to recursive self-improvement and mission analysis.
• Retrieval planning uses structured JSON outputs specifying strategy, targets, tools, and expected outcomes, enabling dynamic selection between vector DB queries, semantic search, and API calls.
• The CompleteAgenticRAG orchestrator automatically routes queries to the appropriate agent layer based on complexity hints and meta-objectives.

Frequently Asked Questions

What is the difference between basic RAG and agentic RAG?

Basic RAG performs a single retrieval step before generation, while agentic RAG implements an autonomous loop that can assess whether retrieved information is sufficient, plan additional retrieval steps, and execute them iteratively until information gaps are closed. According to the davidkimai/context-engineering source code, this is implemented through the BasicRAGAgent class for simple cases and StrategicRAGAgent or MetaCognitiveRAGAgent for complex, multi-step retrieval scenarios.

How does the retrieval planning mechanism work in agentic RAG?

The retrieval planning mechanism uses the plan_next_retrieval method to construct a structured strategy by analyzing current information gaps, available tools (via self.get_available_tools()), and previous attempts (via self.memory.get_previous_attempts()). The planning prompt generates a JSON output specifying the retrieval strategy, target sources, specific tools to invoke (such as semantic_search or vector_db_query), and expected outcomes. This plan is then executed by the agent's retrieval executors in the execute_retrieval method.

What are the three layers of the agentic RAG architecture?

The architecture consists of three layers defined in 02_agentic_rag.md: (1) Prompt Communication – conversational templates like AGENT_REASONING_TEMPLATE and META_COGNITIVE_AGENT_TEMPLATE that structure the reasoning context; (2) Programming Implementation – Python classes including BasicRAGAgent, StrategicRAGAgent, and MetaCognitiveRAGAgent that execute assessment, planning, and synthesis; and (3) Protocol Orchestration – structured protocol shells (e.g., /agent.rag.basic{…}) that formalize steps for chaining, monitoring, and auditing agent actions.

When should I use the MetaCognitiveRAGAgent versus the BasicRAGAgent?

Use BasicRAGAgent for straightforward queries requiring simple gap assessment and single-step retrieval planning, as it implements the core assess_information_state and plan_next_retrieval methods without overhead. Deploy MetaCognitiveRAGAgent for complex, ambiguous, or evolving research questions that require recursive self-improvement, mission analysis, and the ability to evaluate and upgrade its own reasoning modules through the recursive_research_loop method. The CompleteAgenticRAG orchestrator can automatically select the appropriate agent based on complexity hints provided in the query configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}