# How to Build an Agentic RAG System with Query Reasoning and Retrieval Planning

> Build an agentic RAG system using query reasoning and retrieval planning. Discover how to enhance context gathering with an autonomous reasoning loop for superior information retrieval and generation.

- Repository: [davidkimai/context-engineering](https://github.com/davidkimai/context-engineering)
- Tags: tutorial
- Published: 2026-02-28

---

**An agentic RAG system extends traditional retrieval-augmented generation by inserting an autonomous reasoning loop that assesses information gaps, plans retrieval steps, executes them, and adapts strategy until sufficient context is gathered.**

The `davidkimai/context-engineering` repository implements this architecture as a three-layer Software 3.0 stack, providing concrete Python classes and protocol shells that transform static RAG pipelines into dynamic, self-correcting research agents.

## Understanding the Agentic RAG Architecture

Traditional RAG follows a linear path: query, retrieve, generate. Agentic RAG introduces a **cognitive control layer** that treats retrieval as a strategic decision-making process rather than a single function call.

### The Three-Layer Software 3.0 Stack

The repository structures agentic RAG into three distinct layers, each defined in [`00_COURSE/04_retrieval_augmented_generation/02_agentic_rag.md`](https://github.com/davidkimai/context-engineering/blob/main/00_COURSE/04_retrieval_augmented_generation/02_agentic_rag.md):

| Layer | Responsibility | Key Components |
|------|----------------|-----------------|
| **Prompt Communication** | Conversational templates describing current query, known facts, gaps, and retrieval plans. | `AGENT_REASONING_TEMPLATE` (basic), `STRATEGIC_AGENT_TEMPLATE` (intermediate), `META_COGNITIVE_AGENT_TEMPLATE` (advanced) |
| **Programming Implementation** | Python classes that execute assessment, planning, retrieval, and synthesis. | `BasicRAGAgent`, `StrategicRAGAgent`, `MetaCognitiveRAGAgent` |
| **Protocol Orchestration** | Structured protocol shells (e.g., `/agent.rag.basic{…}`) that formalize steps for chaining, monitoring, and auditing. | Protocol definitions throughout the agentic RAG specification |

### Core Architectural Flow

The `CompleteAgenticRAG` class orchestrates the full pipeline through six distinct phases:

1. **Initial Query Reception** – The user query enters the orchestration layer.
2. **Reasoning Session Init** – The system populates `AGENT_REASONING_TEMPLATE` with the query and current context.
3. **Information Assessment** – The LLM returns a structured assessment parsed by `assess_information_state`, evaluating `completeness`, `gaps`, and `confidence`.
4. **Retrieval Planning** – `plan_next_retrieval` crafts a strategy specifying tools, targets, and expected outcomes.
5. **Execution Loop** – The agent iterates (default max 5 iterations) calling `execute_retrieval`, updating the session state until information is sufficient or limits are reached.
6. **Synthesis** – `synthesize_response` generates the final answer, optionally enriched by a knowledge synthesis protocol.

## Implementing Query Reasoning and Retrieval Planning

The transition from basic to agentic RAG centers on two capabilities: **reasoning about information state** and **planning retrieval actions**.

### Information Assessment and Gap Analysis

The `assess_information_state` method (defined in the `BasicRAGAgent` class) parses LLM outputs to determine whether current context satisfies the query. The method evaluates:

- **Completeness**: Percentage of query aspects covered
- **Gaps**: Specific missing information or ambiguities
- **Confidence**: Reliability assessment of current sources

This assessment triggers the planning phase only when gaps are identified, preventing unnecessary retrieval calls for simple queries.

### Dynamic Retrieval Planning

The `plan_next_retrieval` method constructs a structured retrieval strategy by combining:

- **Available tools**: Retrieved via `self.get_available_tools()`
- **Previous attempts**: Accessed through `self.memory.get_previous_attempts()`
- **Current gaps**: From the assessment phase

The planning prompt (lines 165-170 in the specification) instructs the LLM to return a JSON plan:

```json
{
  "strategy": "multi-source semantic search",
  "targets": ["paper-XYZ", "API-docs"],
  "tools": ["semantic_search", "vector_db_query"],
  "expected_outcomes": "relevant excerpts + confidence scores"
}

```

### Execution and Adaptation

The execution loop processes the plan through `execute_retrieval`, which invokes specific tools from the retrieval library. For higher complexity queries, the `StrategicRAGAgent` adds **adaptive execution** through `should_adapt_strategy`, allowing mid-execution re-planning when initial strategies fail.

The `MetaCognitiveRAGAgent` extends this further with `recursive_research_loop`, enabling the agent to evaluate its own research quality and upgrade reasoning modules dynamically.

## Code Implementation: From Basic to Meta-Cognitive Agents

The repository provides concrete Python implementations for each layer of the agentic stack.

### Basic RAG Agent Implementation

The `BasicRAGAgent` class provides the foundational reasoning loop. Located in [`00_COURSE/04_retrieval_augmented_generation/02_agentic_rag.md`](https://github.com/davidkimai/context-engineering/blob/main/00_COURSE/04_retrieval_augmented_generation/02_agentic_rag.md) (lines 107-150), this class implements `assess_information_state`, `plan_next_retrieval`, and `synthesize_response`.

```python
from pathlib import Path
from my_project.agents import BasicRAGAgent, RetrievalTools, ReasoningTemplates

# Load retrieval utilities (vector DB, web search, etc.)

retrieval_tools = RetrievalTools.from_config(Path("config/retrieval.yaml"))

# Load prompt templates

templates = ReasoningTemplates.from_dir(Path("templates/agentic"))

# Instantiate the basic agent

basic_agent = BasicRAGAgent(retrieval_tools, templates)

# Run a query

answer = basic_agent.process_query(
    "What are the main challenges of quantum-aware language models?"
)
print(answer)

```

### Strategic Agent with Multi-Phase Planning

The `StrategicRAGAgent` (lines 298-346) adds mission analysis and adaptive execution for complex, multi-faceted queries.

```python
from my_project.agents import StrategicRAGAgent, RetrievalTools, ReasoningTemplates, StrategyLibrary

tools = RetrievalTools.from_config("config/retrieval.yaml")
templates = ReasoningTemplates.from_dir("templates/strategic")
strategy_lib = StrategyLibrary.load("strategies/strategic.yaml")

strategic_agent = StrategicRAGAgent(tools, templates, strategy_lib)

response = strategic_agent.process_complex_query(
    "Compare the ethical implications of autonomous weapons versus autonomous medical diagnosis systems."
)
print(response)

```

### Meta-Cognitive Agent with Self-Improvement

The `MetaCognitiveRAGAgent` (lines 442-466) implements recursive research loops and self-evaluation for autonomous research tasks.

```python
from my_project.agents import MetaCognitiveRAGAgent, RetrievalTools, ReasoningTemplates, MetaEngine

tools = RetrievalTools.from_config("config/retrieval.yaml")
templates = ReasoningTemplates.from_dir("templates/meta")
meta_engine = MetaEngine.from_config("config/meta_engine.yaml")

meta_agent = MetaCognitiveRAGAgent(tools, templates, meta_engine)

research = meta_agent.conduct_research(
    research_question="Assess the long-term societal impacts of AI-augmented education."
)
print(research["research_findings"])
print("Meta-insights:", research["meta_cognitive_insights"])

```

### Complete System Orchestration

The `CompleteAgenticRAG` class (lines 108-152) orchestrates all three layers, automatically selecting the appropriate agent complexity based on query characteristics.

```python
from my_project.full_system import CompleteAgenticRAG
from my_project.config import SystemConfig

config = SystemConfig.load("config/system.yaml")
rag_system = CompleteAgenticRAG(config)

# The orchestrator automatically picks the appropriate layer

result = rag_system.process_query(
    "Provide a step-by-step plan for deploying a secure, multi-tenant LLM serving platform.",
    complexity_hint="high",
    meta_objectives={"self_improvement": True}
)

print(result)

```

## Configuration and Integration

To wire the agentic RAG system into your infrastructure, configure the retrieval backend and template libraries:

- **Retrieval Indexing**: Configure vector stores, chunking strategies, and metadata filters in [`40_reference/retrieval_indexing.md`](https://github.com/davidkimai/context-engineering/blob/main/40_reference/retrieval_indexing.md) to support the agents' retrieval executors.
- **Cognitive Templates**: Reusable prompt fragments for assessment and planning reside in [`cognitive-tools/cognitive-templates/reasoning.md`](https://github.com/davidkimai/context-engineering/blob/main/cognitive-tools/cognitive-templates/reasoning.md).
- **Program Library**: Utility functions for retrieval, memory, and meta-cognitive evaluation are available in [`cognitive-tools/cognitive-programs/program-library.py`](https://github.com/davidkimai/context-engineering/blob/main/cognitive-tools/cognitive-programs/program-library.py).

All agents accept a `configuration` object bundling retrieval tools, template collections, and optional meta-cognitive engines, enabling plug-and-play deployment across domains (medical, legal, scientific).

## Summary

- **Agentic RAG** transforms static retrieval into an autonomous reasoning loop where the system assesses information gaps, plans retrieval strategies, executes them, and adapts until sufficient context is gathered.
- The **three-layer architecture** (Prompt Communication, Programming Implementation, Protocol Orchestration) provides separation of concerns between conversational templates, Python execution logic, and structured audit shells.
- **Three agent classes**—`BasicRAGAgent`, `StrategicRAGAgent`, and `MetaCognitiveRAGAgent`—offer increasing sophistication, from simple gap assessment to recursive self-improvement and mission analysis.
- **Retrieval planning** uses structured JSON outputs specifying strategy, targets, tools, and expected outcomes, enabling dynamic selection between vector DB queries, semantic search, and API calls.
- The `CompleteAgenticRAG` orchestrator automatically routes queries to the appropriate agent layer based on complexity hints and meta-objectives.

## Frequently Asked Questions

### What is the difference between basic RAG and agentic RAG?

Basic RAG performs a single retrieval step before generation, while agentic RAG implements an autonomous loop that can assess whether retrieved information is sufficient, plan additional retrieval steps, and execute them iteratively until information gaps are closed. According to the `davidkimai/context-engineering` source code, this is implemented through the `BasicRAGAgent` class for simple cases and `StrategicRAGAgent` or `MetaCognitiveRAGAgent` for complex, multi-step retrieval scenarios.

### How does the retrieval planning mechanism work in agentic RAG?

The retrieval planning mechanism uses the `plan_next_retrieval` method to construct a structured strategy by analyzing current information gaps, available tools (via `self.get_available_tools()`), and previous attempts (via `self.memory.get_previous_attempts()`). The planning prompt generates a JSON output specifying the retrieval strategy, target sources, specific tools to invoke (such as `semantic_search` or `vector_db_query`), and expected outcomes. This plan is then executed by the agent's retrieval executors in the `execute_retrieval` method.

### What are the three layers of the agentic RAG architecture?

The architecture consists of three layers defined in [`02_agentic_rag.md`](https://github.com/davidkimai/context-engineering/blob/main/02_agentic_rag.md): (1) **Prompt Communication** – conversational templates like `AGENT_REASONING_TEMPLATE` and `META_COGNITIVE_AGENT_TEMPLATE` that structure the reasoning context; (2) **Programming Implementation** – Python classes including `BasicRAGAgent`, `StrategicRAGAgent`, and `MetaCognitiveRAGAgent` that execute assessment, planning, and synthesis; and (3) **Protocol Orchestration** – structured protocol shells (e.g., `/agent.rag.basic{…}`) that formalize steps for chaining, monitoring, and auditing agent actions.

### When should I use the MetaCognitiveRAGAgent versus the BasicRAGAgent?

Use `BasicRAGAgent` for straightforward queries requiring simple gap assessment and single-step retrieval planning, as it implements the core `assess_information_state` and `plan_next_retrieval` methods without overhead. Deploy `MetaCognitiveRAGAgent` for complex, ambiguous, or evolving research questions that require recursive self-improvement, mission analysis, and the ability to evaluate and upgrade its own reasoning modules through the `recursive_research_loop` method. The `CompleteAgenticRAG` orchestrator can automatically select the appropriate agent based on complexity hints provided in the query configuration.