Difference Between Standard `/api/v1/ask` and Agentic RAG Endpoints

The standard /api/v1/ask endpoint performs single-pass retrieval and generation, while the agentic RAG endpoint orchestrates a multi-step LangGraph workflow with guard-rails, document grading, and automatic query rewriting.

In the jamwithai/production-agentic-rag-course repository, both endpoints share the same base path and return generated answers, but they implement fundamentally different retrieval strategies. Understanding the difference between the standard ask endpoint and agentic RAG endpoint is critical for choosing the right architecture for your use case—whether you need speed and simplicity or robust, self-correcting reasoning.

Core Architectural Differences

The standard endpoint follows a linear pipeline, while the agentic implementation uses a stateful graph architecture.

Standard /api/v1/ask executes a straightforward flow defined in src/routers/ask.py: check cache → run OpenSearch query → build prompt → call Ollama. The core logic lives in the ask_question function (lines 90‑124), which performs a single retrieval pass with no retry mechanism.

Agentic /api/v1/ask-agentic delegates to AgenticRAGService in src/services/agents/agentic_rag.py. This service compiles a LangGraph workflow in its __init__ method (lines 42‑73) that wires together multiple specialized nodes (lines 97‑154): guard-rail validation, adaptive retrieval, document grading, query rewriting, and answer generation.

Key distinctions:

• Retrieval Strategy: The standard endpoint performs one-shot search (BM25 or hybrid) with no refinement. The agentic version starts with a guard-rail check, grades retrieved documents for relevance, and can rewrite the query and retry up to max_retrieval_attempts.
• Safety Controls: The standard endpoint forwards all queries directly to the LLM without validation. The agentic endpoint includes a guard-rail node that evaluates query scope and can short-circuit with an out-of-scope response.
• Document Filtering: Standard RAG feeds all retrieved chunks into the prompt. Agentic RAG grades documents and retains only relevant chunks for generation.
• Observability: Standard logging versus full Langfuse tracing with trace_id and explicit reasoning_steps.

Implementation Deep Dive

Standard RAG Endpoint (Single-Pass Retrieval)

Located in src/routers/ask.py, the standard endpoint is registered as:

ask_router.post("/ask", ...)

The ask_question handler implements a direct retrieval pipeline:

1. Optional cache lookup
2. Single OpenSearch query (supporting hybrid search via use_hybrid parameter)
3. Prompt construction with retrieved context
4. Ollama generation

There is no intermediate validation or retry logic. The method returns answer, sources, chunks_used, and search_mode immediately after the LLM call completes.

Agentic RAG Endpoint (Graph-Based Workflow)

Defined in src/routers/agentic_ask.py and registered at:

router.post("/ask-agentic", ...)

This endpoint instantiates AgenticRAGService and delegates to its ask method. The service builds a compiled graph in _build_graph (lines 97‑154 of src/services/agents/agentic_rag.py) containing these nodes:

• Guard-rail node: Validates query scope before retrieval
• Retrieval node: Performs OpenSearch + embedding search via tools defined in src/services/agents/tools.py
• Grading node: Evaluates document relevance using LLM-as-a-judge
• Rewrite node: Generates improved queries when initial retrieval returns irrelevant chunks
• Answer node: Generates final response from filtered context

The graph supports cyclic execution: if grading determines documents are irrelevant, the flow routes back to the rewrite node, then performs additional retrieval attempts until reaching max_retrieval_attempts or finding relevant context.

Response Formats and Observability

The standard endpoint returns minimal metadata:

{
  "query": "...",
  "answer": "...",
  "sources": ["..."],
  "chunks_used": 4,
  "search_mode": "hybrid"
}

The agentic endpoint returns the same fields plus execution metadata:

{
  "query": "...",
  "answer": "...",
  "sources": [{"url": "...", "title": "..."}],
  "chunks_used": 3,
  "search_mode": "hybrid",
  "reasoning_steps": [
    "Validated query scope (score: 92/100)",
    "Retrieved documents (1 attempt(s))",
    "Graded documents (2 relevant)",
    "Generated answer from context"
  ],
  "retrieval_attempts": 1,
  "trace_id": "tr-abc123"
}

The trace_id enables full traceability in Langfuse, capturing every LLM call and node transition, while reasoning_steps provides transparency into the agent's decision process.

Code Examples

Calling the Standard Endpoint

curl -X POST "http://localhost:8000/api/v1/ask" \
  -H "Content-Type: application/json" \
  -d '{
        "query": "What are the latest advances in transformer architectures?",
        "top_k": 5,
        "use_hybrid": true,
        "model": "llama3.1:8b"
      }'

Calling the Agentic Endpoint

curl -X POST "http://localhost:8000/api/v1/ask-agentic" \
  -H "Content-Type: application/json" \
  -d '{
        "query": "Explain the safety concerns of large language models.",
        "top_k": 5,
        "use_hybrid": true,
        "model": "llama3.1:8b"
      }'

Streaming with Standard RAG

The standard router also exposes a streaming endpoint for real-time token delivery:

curl -N -X POST "http://localhost:8000/api/v1/stream" \
  -H "Content-Type: application/json" \
  -d '{"query":"Summarize the key ideas of the paper Attention Is All You Need","top_k":3}'

This returns server-sent events with incremental chunks and a final done flag, functionality not currently implemented in the agentic workflow.

Key Source Files

Understanding the repository structure helps navigate the implementation differences:

• src/routers/ask.py – Defines the standard /api/v1/ask and streaming routes with the ask_question handler
• src/routers/agentic_ask.py – Exposes /api/v1/ask-agentic and forwards requests to the graph service
• src/services/agents/agentic_rag.py – Implements the AgenticRAGService with LangGraph workflow compilation and node orchestration
• src/services/agents/nodes.py – Contains concrete async node functions (guard-rail, grading, rewriting) referenced by the graph
• src/services/agents/tools.py – Supplies the OpenSearch retriever used by both endpoints
• src/schemas/api/ask.py – Defines AskRequest, AskResponse, and AgenticAskResponse Pydantic models shared across both implementations

Summary

• Use /api/v1/ask when you need fast, single-pass retrieval-augmented generation without complex safety checks or retry logic
• Use /api/v1/ask-agentic when you require guard-rail validation, automatic query refinement, document relevance grading, and detailed reasoning traces
• The standard endpoint implements linear logic in src/routers/ask.py, while the agentic version uses a compiled LangGraph state machine in src/services/agents/agentic_rag.py
• Agentic RAG provides superior observability via reasoning_steps and Langfuse trace_id, but adds latency due to multi-node execution

Frequently Asked Questions

What triggers the query rewriting in the agentic endpoint?

The rewrite node activates when the document grading node determines that retrieved chunks are irrelevant to the query. According to the graph implementation in src/services/agents/agentic_rag.py, the workflow checks document relevance after each retrieval attempt; if the score falls below threshold and retrieval_attempts is less than max_retrieval_attempts, the graph routes to the rewrite node to generate an optimized query before retrying retrieval.

Can I use hybrid search with both endpoints?

Yes. Both the standard ask_question handler and the agentic retrieval node accept a use_hybrid parameter that enables combined BM25 and embedding search against the OpenSearch index. The standard endpoint passes this directly to the search client, while the agentic workflow passes it through the tool definition in src/services/agents/tools.py.

Why does the agentic endpoint have a guard-rail while the standard endpoint does not?

The agentic workflow in src/services/agents/agentic_rag.py explicitly includes a guard-rail node as the first graph step to prevent compute waste on out-of-scope queries and to provide safety controls before retrieval. The standard endpoint in src/routers/ask.py prioritizes low latency and simplicity, forwarding all queries directly to the LLM without intermediate validation gates.

How do I add custom nodes to the agentic workflow?

Because the agentic implementation uses a graph-based architecture defined in _build_graph (lines 97‑154 of src/services/agents/agentic_rag.py), you can extend the workflow by adding new node functions to src/services/agents/nodes.py, then wire them into the graph compilation in AgenticRAGService.__init__. This node-driven design allows you to modify retrieval logic, add fact-checking steps, or implement citation handling without changing the endpoint code in src/routers/agentic_ask.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}