# How to Configure Guardrail Threshold and Max Retrieval Attempts in Agentic RAG

> Configure guardrail threshold and max retrieval attempts in Agentic RAG using GraphConfig. Control query scope and retrieval retries effortlessly for optimized performance. Learn how now.

- Repository: [jamwithai/production-agentic-rag-course](https://github.com/jamwithai/production-agentic-rag-course)
- Tags: how-to-guide
- Published: 2026-03-23

---

**Set `guardrail_threshold` and `max_retrieval_attempts` in the `GraphConfig` class to control query scope validation and retrieval retry limits, with values automatically propagated through the LangGraph runtime `Context` object to enforcement nodes.**

Agentic RAG (Retrieval-Augmented Generation) pipelines require precise boundaries for safety and resource management. In the `jamwithai/production-agentic-rag-course` repository, two configuration parameters govern these behaviors: `guardrail_threshold` validates query relevance before processing, while `max_retrieval_attempts` caps the number of document retrieval retries. Both settings are defined in `GraphConfig` and injected into the immutable `Context` object that LangGraph passes to every node in the workflow.

## Configuration Architecture

The system uses a centralized configuration pattern where `GraphConfig` declares defaults, and `Context` carries runtime values to individual nodes.

### GraphConfig Declaration

The `GraphConfig` class in [`src/services/agents/config.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/config.py) declares both parameters with type-safe defaults using Pydantic:

```python
class GraphConfig(BaseModel):
    guardrail_threshold: int = 60   # Minimum score (0-100)

    max_retrieval_attempts: int = 2 # Upper bound on retries

```

These defaults enforce a moderate safety envelope: queries must score at least 60 on the guardrail evaluation, and the system will attempt retrieval a maximum of two times before falling back.

### Runtime Context Injection

When `AgenticRAGService` initializes the workflow in [`src/services/agents/agentic_rag.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/agentic_rag.py), it creates a `Context` object that injects configuration values into the LangGraph runtime:

```python
runtime_context = Context(
    ...,
    guardrail_threshold=self.graph_config.guardrail_threshold,
    max_retrieval_attempts=self.graph_config.max_retrieval_attempts,
    ...
)

```

The `Context` class defined in [`src/services/agents/context.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/context.py) serves as an immutable container that LangGraph automatically passes to every node, ensuring type-safe access to configuration without global state.

## Configuring the Guardrail Threshold

The `guardrail_threshold` parameter determines whether a user query is considered in-scope based on LLM scoring.

### Guardrail Evaluation Flow

In [`src/services/agents/nodes/guardrail_node.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/nodes/guardrail_node.py), the node executes a guardrail LLM prompt and compares the returned score against the injected threshold:

```python
score = guardrail_result.score
threshold = runtime.context.guardrail_threshold
return "continue" if score >= threshold else "out_of_scope"

```

If the score meets or exceeds the configured threshold, the graph routes to the "continue" edge for retrieval; otherwise, it routes to the `out_of_scope` node, terminating the request with a polite refusal.

### Adjusting the Threshold

Raise the threshold to enforce stricter scope validation, or lower it to allow more permissive query handling. Values range from 0 to 100, where higher numbers require stronger relevance signals from the guardrail LLM.

## Configuring Max Retrieval Attempts

The `max_retrieval_attempts` parameter prevents infinite loops by limiting how many times the system attempts to retrieve documents before returning a fallback message.

### Retrieval Enforcement Logic

In [`src/services/agents/nodes/retrieve_node.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/nodes/retrieve_node.py), the node checks the current attempt count against the configured maximum:

```python
max_attempts = runtime.context.max_retrieval_attempts
if current_attempts >= max_attempts:
    # Return fallback AIMessage explaining no results found

else:
    new_attempt_count = current_attempts + 1
    # Issue tool call to fetch papers

```

Once the limit is reached, the system returns a verbose fallback message guiding users to rephrase their query, preventing resource exhaustion from unsuccessful retrievals.

## Practical Configuration Methods

You can configure these parameters programmatically, via environment variables, or inspect them at runtime.

### Programmatic Configuration

Override defaults by passing a custom `GraphConfig` when instantiating `AgenticRAGService`:

```python
from src.services.agents.agentic_rag import AgenticRAGService
from src.services.agents.config import GraphConfig

cfg = GraphConfig(
    guardrail_threshold=75,      # Require higher relevance

    max_retrieval_attempts=3,   # Allow one extra retry

)

service = AgenticRAGService(
    opensearch_client=my_os_client,
    ollama_client=my_ollama,
    embeddings_client=my_jina,
    graph_config=cfg,
)

```

This approach binds configuration to the specific service instance without affecting other deployments.

### Environment Variable Overrides

Using pydantic-settings, any `GraphConfig` field can be overridden with environment variables using the `AGENTIC_RAG__` prefix (the `__` delimiter is defined in [`src/config.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/config.py)):

```bash
export AGENTIC_RAG__GUARDRAIL_THRESHOLD=80
export AGENTIC_RAG__MAX_RETRIEVAL_ATTEMPTS=4

```

These values are automatically loaded when `get_settings()` initializes, and `GraphConfig` picks them up without requiring code changes.

### Runtime Verification

Inspect current configuration values through the service instance:

```python
print("Guardrail threshold:", service.graph_config.guardrail_threshold)
print("Max retrieval attempts:", service.graph_config.max_retrieval_attempts)

```

Additionally, `AgenticRAGService.__init__` logs both parameters at startup for operational verification:

```python
logger.info(f"  Guardrail threshold: {self.graph_config.guardrail_threshold}")
logger.info(f"  Max retrieval attempts: {self.graph_config.max_retrieval_attempts}")

```

## Summary

- **Guardrail threshold** is configured via `GraphConfig.guardrail_threshold` (default: `60`) and enforced in [`src/services/agents/nodes/guardrail_node.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/nodes/guardrail_node.py) to validate query scope before processing.
- **Max retrieval attempts** is configured via `GraphConfig.max_retrieval_attempts` (default: `2`) and enforced in [`src/services/agents/nodes/retrieve_node.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/nodes/retrieve_node.py) to prevent infinite retrieval loops.
- Both values are injected into the LangGraph `Context` object at runtime in [`src/services/agents/agentic_rag.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/agentic_rag.py) and passed automatically to nodes.
- Configuration can be set programmatically via `AgenticRAGService` instantiation or overridden using `AGENTIC_RAG__*` environment variables parsed by pydantic-settings.
- The `jamwithai/production-agentic-rag-course` implementation logs both parameters at startup in `AgenticRAGService.__init__` for operational visibility.

## Frequently Asked Questions

### What is the default guardrail threshold in agentic RAG?

The default `guardrail_threshold` is `60`, defined in [`src/services/agents/config.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/config.py) within the `GraphConfig` class. This means queries must score at least 60 out of 100 on the guardrail LLM evaluation in [`src/services/agents/nodes/guardrail_node.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/nodes/guardrail_node.py) to be considered in-scope and proceed to retrieval.

### How does max_retrieval_attempts prevent infinite loops?

The `max_retrieval_attempts` parameter caps the number of retrieval retries at `2` by default. In [`src/services/agents/nodes/retrieve_node.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/nodes/retrieve_node.py), the node compares the current attempt count against this limit. If the threshold is reached without successful retrieval, the system returns a fallback message instead of continuing to loop, ensuring resources are not exhausted on unanswerable queries.

### Can I override these settings without modifying code?

Yes. The repository uses pydantic-settings to enable environment variable overrides. Prefix any `GraphConfig` field with `AGENTIC_RAG__` (e.g., `AGENTIC_RAG__GUARDRAIL_THRESHOLD=85`). These variables are automatically parsed when the service initializes in [`src/services/agents/agentic_rag.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/agentic_rag.py), allowing configuration changes without code deployment.

### Where can I verify the active configuration values?

Active values are logged by `AgenticRAGService` during initialization in [`src/services/agents/agentic_rag.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/src/services/agents/agentic_rag.py). You can also inspect `service.graph_config.guardrail_threshold` and `service.graph_config.max_retrieval_attempts` programmatically at runtime, or check the unit tests in [`tests/unit/services/agents/test_agentic_rag.py`](https://github.com/jamwithai/production-agentic-rag-course/blob/main/tests/unit/services/agents/test_agentic_rag.py) which verify these defaults.