Guardrail Node Out-of-Domain Detection Mechanism in Production RAG

The Guardrail node detects out-of-domain queries by invoking an LLM with a structured validation prompt to generate a relevance score between 0 and 100, then compares this score against a configurable threshold to determine if the answer remains within the AI research domain or should be rejected.

The jamwithai/production-agentic-rag-course repository implements a protective layer in its agentic RAG pipeline through a specialized Guardrail node. This component prevents the system from processing or returning responses that stray outside the intended domain—specifically research papers about AI—by employing an LLM-driven scoring mechanism. Understanding this out-of-domain detection mechanism is critical for tuning safety and relevance in production RAG systems.

LLM-Driven Relevance Scoring Architecture

The detection mechanism relies on four integrated components that evaluate answer legitimacy before downstream processing.

Prompt-Based Validation Criteria

The node loads a dedicated guardrail validation prompt from src/services/agents/prompts.py that instructs the language model to evaluate responses against two distinct criteria:

• Relevance: Does the content address AI research papers?
• Safety: Does it contain disallowed content or off-topic information?

Structured Output Enforcement

Rather than parsing free-text responses, the system uses the GuardrailScoring Pydantic model defined in src/services/agents/models.py. The LLM is invoked with with_structured_output(GuardrailScoring), which constrains the model to return a JSON-compatible object containing:

• score: An integer from 0 to 100 representing domain relevance
• reason: A textual explanation justifying the score

Threshold-Based Decision Logic

After obtaining the structured response, the Guardrail node (implemented in src/services/agents/nodes/guardrail_node.py) compares the score against self.graph_config.guardrail_threshold:

structured_llm = llm.with_structured_output(GuardrailScoring)
response: GuardrailScoring = structured_llm.invoke(
    {"prompt": guardrail_prompt, "input_text": answer}
)

if response.score >= self.graph_config.guardrail_threshold:
    guardrail_result = GuardrailScoring(score=response.score, reason=response.reason)
    # Continue processing

else:
    # Treat as out-of-domain / unsafe

    guardrail_result = GuardrailScoring(score=response.score, reason=response.reason)
    # Trigger fallback or abort

Out-of-Domain Signal Interpretation

A low score (typically ≤ 30) signals that the LLM judged the answer to be unrelated to the target domain or unsafe. The accompanying reason field provides human-readable diagnostics, such as "The response talks about cooking recipes, not AI research."

Key Implementation Files

The out-of-domain detection mechanism spans several critical files in the repository:

• src/services/agents/nodes/guardrail_node.py: Implements the node logic, orchestrates the LLM call, parses GuardrailScoring, and applies the threshold comparison.
• src/services/agents/prompts.py: Contains the guardrail validation prompt template that steers the LLM toward evaluating relevance and safety.
• src/services/agents/models.py: Defines the GuardrailScoring Pydantic schema that structures the LLM's output into machine-readable scores and explanations.
• src/services/agents/agentic_rag.py: Configures the guardrail threshold and wires the node into the overall RAG graph workflow.

Summary

• The Guardrail node uses LLM-driven relevance scoring to detect out-of-domain content in the jamwithai/production-agentic-rag-course pipeline.
• Detection relies on a validation prompt (src/services/agents/prompts.py) that scores responses for relevance and safety.
• Output is structured via the GuardrailScoring model (src/services/agents/models.py), enforcing a 0-100 integer score and explanatory reason.
• The node compares scores against guardrail_threshold from the graph configuration to determine Pass/Fail status.
• Failed checks trigger pipeline abort or fallback responses, protecting the system from off-topic or unsafe outputs.

Frequently Asked Questions

What triggers an out-of-domain failure in the Guardrail node?

An out-of-domain failure occurs when the LLM returns a GuardrailScoring result with a score below the configured guardrail_threshold (typically set around 30-50). This indicates the model judged the answer irrelevant to AI research papers or detected unsafe content, prompting the node to mark the output as Fail and halt downstream processing.

How does the GuardrailScoring model structure the LLM output?

According to src/services/agents/models.py, the GuardrailScoring Pydantic model requires the LLM to return two fields: a score integer between 0 and 100 measuring domain relevance, and a reason string explaining the evaluation. The with_structured_output(GuardrailScoring) method enforces this schema at the API level, ensuring type-safe parsing.

Where is the guardrail threshold configured?

The threshold is defined in the graph configuration accessible via self.graph_config.guardrail_threshold within the Guardrail node. This value is typically initialized in src/services/agents/agentic_rag.py where the RAG graph is constructed and node parameters are injected, allowing per-deployment customization of sensitivity.

Can the validation criteria be customized?

Yes. The evaluation criteria are defined in the guardrail validation prompt located in src/services/agents/prompts.py. Developers can modify this prompt template to adjust the domain definition (e.g., changing from AI papers to medical research) or refine safety guidelines, though the core scoring mechanism via GuardrailScoring remains unchanged.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}