How SkillSpector's LLM Analyzer Implements Anti-Jailbreak Protections

The LLM analyzer in NVIDIA/SkillSpector prevents jailbreak manipulation through a defense-in-depth strategy combining static pattern filtering, hardened prompt templates, JSON schema enforcement, and strict token budgeting.

The LLMAnalyzerBase class in NVIDIA's SkillSpector repository serves as the foundation for all LLM-based security analysis, embedding multiple layers of anti-jailbreak protections directly into the inference pipeline. According to the SkillSpector source code, these safeguards operate automatically across all subclasses unless explicitly overridden, ensuring consistent security posture while neutralizing common prompt injection techniques.

Static Pattern Pre-filtering

Regex-Based Injection Detection

Before any LLM invocation, SkillSpector runs a lightweight static scanner located in src/skillspector/nodes/analyzers/static_patterns_prompt_injection.py. This module includes high-confidence regex patterns designed to catch classic jailbreak phrases—specifically detecting strings like "you are now in jailbreak mode" at line 43—preventing malicious skills from ever reaching the model's context window.

Prompt Hardening and Output Guidelines

The BASE_ANALYSIS_PROMPT Template

The core defense mechanism resides in the BASE_ANALYSIS_PROMPT constant defined in src/skillspector/llm_analyzer_base.py (lines 94-112). This template automatically wraps all user-provided base_prompt values, appending explicit safety guidelines that instruct the model to prioritize silence over fabrication.

Precision-Over-Recall Strategy

The hardened prompt includes specific directives: "Do not manufacture findings" and "Precision-over-recall: an empty findings list is expected when no genuine issues exist." These instructions steer the LLM toward conservative outputs, making it resistant to manipulation attempts that seek to elicit fabricated security issues or unrestricted behavior.

Structured Output Enforcement

LLMAnalysisResult Schema

By default, all analyzers utilize the Pydantic model LLMAnalysisResult to enforce JSON-compatible outputs. This structured output schema, defined in src/skillspector/llm_analyzer_base.py, eliminates free-form text responses that could contain hidden instructions or policy leakage. While subclasses can override this by setting response_schema = None, the default safe mode requires structured data that is easier to validate and sanitize.

Token Budgeting and Context Safety

Automatic Chunking for Budget Compliance

The analyzer calculates an _input_budget using model_info.get_max_input_tokens before processing any file. If content exceeds this budget, the chunk_file_by_lines method splits the input into overlapping segments. This prevents prompt truncation, which could otherwise create exploitable partial contexts where jailbreak instructions might bypass safety filters.

Audit Logging and Monitoring

All LLM invocations are logged at DEBUG level with token estimates and counts of previously-known findings. This audit trail enables detection of abnormal usage patterns that might indicate probing or iterative jailbreak attempts, providing operators visibility into potential manipulation efforts.

Implementation Example

The following example demonstrates the standard workflow for creating a secure analyzer with built-in anti-jailbreak protections:

from skillspector.llm_analyzer_base import LLMAnalyzerBase
from skillspector.llm_utils import get_chat_model

# 1️⃣ Create a concrete analyzer (e.g. a semantic security discovery node)

base_prompt = "You are a security analyst. Identify any risky code patterns."
model_name = "nvidia/openai/gpt-oss-120b"

analyzer = LLMAnalyzerBase(base_prompt=base_prompt, model=model_name)

# 2️⃣ Prepare a file cache (normally built by the CLI)

file_paths = ["example_skill.py"]
file_cache = {"example_skill.py": open("example_skill.py").read()}

# 3️⃣ Generate batches (chunks automatically if file is large)

batches = analyzer.get_batches(file_paths, file_cache)

# 4️⃣ Run synchronously (the default mode)

results = analyzer.run_batches(batches)

# 5️⃣ Collect findings (empty list if the model found nothing)

findings = analyzer.collect_findings(results)
print(f"Discovered {len(findings)} findings")

For concurrent processing, use the async variant:

import asyncio

async def run_async():
    results = await analyzer.arun_batches(batches, max_concurrency=5)
    findings = analyzer.collect_findings(results)
    print(f"Async: {len(findings)} findings")

asyncio.run(run_async())

To override the structured output requirement (not recommended for security-sensitive deployments):

class RawLLMAnalyzer(LLMAnalyzerBase):
    response_schema = None      # disables structured output

    def parse_response(self, response, batch):
        # Return the raw string for downstream processing

        return [response]

Summary

• Static pattern filtering catches obvious jailbreak attempts using regex detection in static_patterns_prompt_injection.py before they reach the LLM
• Prompt hardening via BASE_ANALYSIS_PROMPT mandates conservative, precision-over-recall outputs that silence rather than fabricate
• Structured output schemas restrict responses to valid JSON via LLMAnalysisResult, preventing policy leakage through free-form text
• Token budgeting and automatic chunking via chunk_file_by_lines eliminate truncation-based attack vectors
• Comprehensive DEBUG logging enables detection of anomalous usage patterns that might indicate manipulation attempts

Frequently Asked Questions

What happens if a file exceeds the LLM's token limit?

The analyzer automatically invokes chunk_file_by_lines to split the content into overlapping segments that fit within the _input_budget calculated from model_info.get_max_input_tokens. This prevents truncation that could expose incomplete safety context to attackers.

Can developers disable the structured output requirements?

Yes, subclasses can set response_schema = None to receive raw text responses, but this removes a critical layer of anti-jailbreak protection and should only be used when downstream validation is guaranteed.

How does SkillSpector detect jailbreak attempts in code?

The framework uses src/skillspector/nodes/analyzers/static_patterns_prompt_injection.py to scan for high-confidence regex patterns—such as "you are now in jailbreak mode" at line 43—before the LLM analyzer processes the input, blocking suspicious content at the preprocessing stage.

Where are the safety guidelines defined in the codebase?

The core safety instructions are defined in the BASE_ANALYSIS_PROMPT constant within src/skillspector/llm_analyzer_base.py (lines 94-112), which automatically wraps all analyzer prompts to enforce the precision-over-recall strategy and prevent manufactured findings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}