How to Configure Guardrail Threshold and Max Retrieval Attempts in Agentic RAG

Set guardrail_threshold and max_retrieval_attempts in the GraphConfig class to control query scope validation and retrieval retry limits, with values automatically propagated through the LangGraph runtime Context object to enforcement nodes.

Agentic RAG (Retrieval-Augmented Generation) pipelines require precise boundaries for safety and resource management. In the jamwithai/production-agentic-rag-course repository, two configuration parameters govern these behaviors: guardrail_threshold validates query relevance before processing, while max_retrieval_attempts caps the number of document retrieval retries. Both settings are defined in GraphConfig and injected into the immutable Context object that LangGraph passes to every node in the workflow.

Configuration Architecture

The system uses a centralized configuration pattern where GraphConfig declares defaults, and Context carries runtime values to individual nodes.

GraphConfig Declaration

The GraphConfig class in src/services/agents/config.py declares both parameters with type-safe defaults using Pydantic:

class GraphConfig(BaseModel):
    guardrail_threshold: int = 60   # Minimum score (0-100)

    max_retrieval_attempts: int = 2 # Upper bound on retries

These defaults enforce a moderate safety envelope: queries must score at least 60 on the guardrail evaluation, and the system will attempt retrieval a maximum of two times before falling back.

Runtime Context Injection

When AgenticRAGService initializes the workflow in src/services/agents/agentic_rag.py, it creates a Context object that injects configuration values into the LangGraph runtime:

runtime_context = Context(
    ...,
    guardrail_threshold=self.graph_config.guardrail_threshold,
    max_retrieval_attempts=self.graph_config.max_retrieval_attempts,
    ...
)

The Context class defined in src/services/agents/context.py serves as an immutable container that LangGraph automatically passes to every node, ensuring type-safe access to configuration without global state.

Configuring the Guardrail Threshold

The guardrail_threshold parameter determines whether a user query is considered in-scope based on LLM scoring.

Guardrail Evaluation Flow

In src/services/agents/nodes/guardrail_node.py, the node executes a guardrail LLM prompt and compares the returned score against the injected threshold:

score = guardrail_result.score
threshold = runtime.context.guardrail_threshold
return "continue" if score >= threshold else "out_of_scope"

If the score meets or exceeds the configured threshold, the graph routes to the "continue" edge for retrieval; otherwise, it routes to the out_of_scope node, terminating the request with a polite refusal.

Adjusting the Threshold

Raise the threshold to enforce stricter scope validation, or lower it to allow more permissive query handling. Values range from 0 to 100, where higher numbers require stronger relevance signals from the guardrail LLM.

Configuring Max Retrieval Attempts

The max_retrieval_attempts parameter prevents infinite loops by limiting how many times the system attempts to retrieve documents before returning a fallback message.

Retrieval Enforcement Logic

In src/services/agents/nodes/retrieve_node.py, the node checks the current attempt count against the configured maximum:

max_attempts = runtime.context.max_retrieval_attempts
if current_attempts >= max_attempts:
    # Return fallback AIMessage explaining no results found

else:
    new_attempt_count = current_attempts + 1
    # Issue tool call to fetch papers

Once the limit is reached, the system returns a verbose fallback message guiding users to rephrase their query, preventing resource exhaustion from unsuccessful retrievals.

Practical Configuration Methods

You can configure these parameters programmatically, via environment variables, or inspect them at runtime.

Programmatic Configuration

Override defaults by passing a custom GraphConfig when instantiating AgenticRAGService:

from src.services.agents.agentic_rag import AgenticRAGService
from src.services.agents.config import GraphConfig

cfg = GraphConfig(
    guardrail_threshold=75,      # Require higher relevance

    max_retrieval_attempts=3,   # Allow one extra retry

)

service = AgenticRAGService(
    opensearch_client=my_os_client,
    ollama_client=my_ollama,
    embeddings_client=my_jina,
    graph_config=cfg,
)

This approach binds configuration to the specific service instance without affecting other deployments.

Environment Variable Overrides

Using pydantic-settings, any GraphConfig field can be overridden with environment variables using the AGENTIC_RAG__ prefix (the __ delimiter is defined in src/config.py):

export AGENTIC_RAG__GUARDRAIL_THRESHOLD=80
export AGENTIC_RAG__MAX_RETRIEVAL_ATTEMPTS=4

These values are automatically loaded when get_settings() initializes, and GraphConfig picks them up without requiring code changes.

Runtime Verification

Inspect current configuration values through the service instance:

print("Guardrail threshold:", service.graph_config.guardrail_threshold)
print("Max retrieval attempts:", service.graph_config.max_retrieval_attempts)

Additionally, AgenticRAGService.__init__ logs both parameters at startup for operational verification:

logger.info(f"  Guardrail threshold: {self.graph_config.guardrail_threshold}")
logger.info(f"  Max retrieval attempts: {self.graph_config.max_retrieval_attempts}")

Summary

• Guardrail threshold is configured via GraphConfig.guardrail_threshold (default: 60) and enforced in src/services/agents/nodes/guardrail_node.py to validate query scope before processing.
• Max retrieval attempts is configured via GraphConfig.max_retrieval_attempts (default: 2) and enforced in src/services/agents/nodes/retrieve_node.py to prevent infinite retrieval loops.
• Both values are injected into the LangGraph Context object at runtime in src/services/agents/agentic_rag.py and passed automatically to nodes.
• Configuration can be set programmatically via AgenticRAGService instantiation or overridden using AGENTIC_RAG__* environment variables parsed by pydantic-settings.
• The jamwithai/production-agentic-rag-course implementation logs both parameters at startup in AgenticRAGService.__init__ for operational visibility.

Frequently Asked Questions

What is the default guardrail threshold in agentic RAG?

The default guardrail_threshold is 60, defined in src/services/agents/config.py within the GraphConfig class. This means queries must score at least 60 out of 100 on the guardrail LLM evaluation in src/services/agents/nodes/guardrail_node.py to be considered in-scope and proceed to retrieval.

How does max_retrieval_attempts prevent infinite loops?

The max_retrieval_attempts parameter caps the number of retrieval retries at 2 by default. In src/services/agents/nodes/retrieve_node.py, the node compares the current attempt count against this limit. If the threshold is reached without successful retrieval, the system returns a fallback message instead of continuing to loop, ensuring resources are not exhausted on unanswerable queries.

Can I override these settings without modifying code?

Yes. The repository uses pydantic-settings to enable environment variable overrides. Prefix any GraphConfig field with AGENTIC_RAG__ (e.g., AGENTIC_RAG__GUARDRAIL_THRESHOLD=85). These variables are automatically parsed when the service initializes in src/services/agents/agentic_rag.py, allowing configuration changes without code deployment.

Where can I verify the active configuration values?

Active values are logged by AgenticRAGService during initialization in src/services/agents/agentic_rag.py. You can also inspect service.graph_config.guardrail_threshold and service.graph_config.max_retrieval_attempts programmatically at runtime, or check the unit tests in tests/unit/services/agents/test_agentic_rag.py which verify these defaults.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}