How SkillSpector's LangGraph Two-Stage Analysis Pipeline Works: Static Detection Meets LLM Validation

SkillSpector executes a two-stage LangGraph pipeline where static analyzers first scan code for vulnerabilities, then an LLM meta-analyzer filters false positives and enriches findings with remediation guidance before generating the final security report.

SkillSpector, NVIDIA's open-source security scanner for AI skills, leverages LangGraph to orchestrate a sophisticated two-stage analysis pipeline. This architecture separates fast, rule-based detection from computationally expensive AI validation, enabling efficient scanning of skill source code while maintaining high accuracy. The entire workflow is defined in src/skillspector/graph.py and processes code through distinct static and LLM-driven phases.

Stage 1: Static Analysis (Rule-Based Detection)

The first stage performs exhaustive, file-system-based scanning without external API calls. This ensures rapid feedback and identifies potential vulnerability patterns across the entire skill source tree.

Input Resolution and Context Building

The pipeline begins with two initialization nodes that prepare the execution environment. The resolve_input node (implemented in skillspector/nodes/resolve_input.py) normalizes user-provided paths, handling URLs, zip files, or local directories, and populates state["skill_path"] along with a temporary cleanup directory.

Subsequently, build_context (in skillspector/nodes/build_context.py) traverses the skill directory to construct a comprehensive file cache and extract component metadata. This node also parses the optional skill.md manifest, ensuring downstream analyzers have structured access to the codebase.

Analyzer Node Execution

Following context building, the graph executes all static analyzer nodes registered in ANALYZER_NODE_IDS (defined in skillspector/nodes/analyzers/__init__.py). Each node implements a specific security check:

• Prompt injection pattern detection
• Unsafe file-system call identification
• Behavioral taint tracking

Every analyzer receives the full state object, reads from state["file_cache"], and appends Finding objects to state["findings"]. This parallel-sequential execution ensures comprehensive coverage while maintaining shared state consistency.

Stage 2: LLM-Driven Filtering and Enrichment

Once static analysis completes, the meta_analyzer node (in skillspector/nodes/meta_analyzer.py) initiates the second stage. This phase uses the LLMMetaAnalyzer class to validate and enrich the raw findings.

Batching and Asynchronous Processing

The meta-analyzer organizes findings into per-file batches using LLMMetaAnalyzer.get_batches(). For each file containing at least one alert, the system makes an asynchronous LLM call. The prompt engineering and token-budget management are handled by the LLMAnalyzerBase parent class, ensuring consistent API utilization.

The LLM evaluates each finding against a Pydantic schema (MetaAnalyzerResult), determining:

• Vulnerability validity (is_vulnerability boolean)
• Confidence score (0.0 to 1.0)
• Risk explanation and remediation steps
• Intent classification and impact assessment

Validation and Confidence Filtering

The apply_filter method processes LLM responses and enforces strict quality thresholds. Only findings meeting the following criteria survive:

• is_vulnerability must be True
• Confidence score must be ≥ 0.6

Findings failing these criteria are discarded as false positives. Valid findings are enriched with generated explanations and remediation guidance, then stored in state["filtered_findings"]. Missing fields receive default text to ensure report completeness.

Report Generation and Risk Scoring

The final report node (in skillspector/nodes/report.py) consumes the enriched findings to produce user-consumable output. This node computes a quantitative risk score using the formula:

• CRITICAL: +50 points
• HIGH: +25 points
• MEDIUM: +10 points
• LOW: +5 points

If executable scripts are detected in the skill, the total score is multiplied by 1.3. The node then renders output in SARIF, JSON, Markdown, or terminal formats based on configuration.

Pipeline Orchestration in LangGraph

The complete workflow is assembled in create_graph() within src/skillspector/graph.py. The StateGraph wires nodes and edges to enforce the two-stage execution order:

from skillspector.graph import create_graph
from langgraph.graph import START, END

def create_graph():
    workflow = StateGraph(SkillspectorState)
    
    # Core nodes

    workflow.add_node("resolve_input", resolve_input)
    workflow.add_node("build_context", build_context)
    workflow.add_node("meta_analyzer", meta_analyzer)
    workflow.add_node("report", report)
    
    # Static analyzer nodes

    for analyzer_id in ANALYZER_NODE_IDS:
        workflow.add_node(analyzer_id, ANALYZER_NODES[analyzer_id])
    
    # Edge wiring

    workflow.add_edge(START, "resolve_input")
    workflow.add_edge("resolve_input", "build_context")
    
    for analyzer_id in ANALYZER_NODE_IDS:
        workflow.add_edge("build_context", analyzer_id)
        workflow.add_edge(analyzer_id, "meta_analyzer")
    
    workflow.add_edge("meta_analyzer", "report")
    workflow.add_edge("report", END)
    
    return workflow.compile()

The compiled graph executes deterministically: static analyzers run after context building, the meta-analyzer waits for all static findings, and the report node finalizes the output.

Running the Pipeline

Command-Line Execution

Invoke the complete two-stage pipeline via CLI:

# Full analysis with LLM validation

skillspector scan ./my_skill --use-llm --output-format json

# Static analysis only (offline mode)

skillspector scan ./my_skill --no-llm

When --no-llm is specified, the meta_analyzer detects this flag and returns raw findings through a _fallback_filtered wrapper, bypassing API calls.

Programmatic Invocation

Integrate the pipeline directly into Python applications:

from skillspector.graph import graph
from skillspector.state import SkillspectorState

# Initialize state with target path

state = SkillspectorState(
    input_path="https://github.com/example/skill.git",
    use_llm=True
)

# Execute two-stage pipeline

final_state = graph.invoke(state)

# Access enriched findings

enriched = final_state["filtered_findings"]
print(f"LLM-validated vulnerabilities: {len(enriched)}")

The graph.invoke() call blocks until all nodes complete, returning the final state containing filtered_findings and the generated report.

Summary

• SkillSpector implements a LangGraph-based two-stage analysis pipeline that separates static detection from LLM validation.
• Stage 1 uses rule-based analyzers registered in ANALYZER_NODE_IDS to scan files and populate findings without external API dependencies.
• Stage 2 employs LLMMetaAnalyzer to batch process findings asynchronously, filtering for confidence ≥ 0.6 and enriching true positives with remediation guidance.
• The report node computes risk scores (CRITICAL +50, HIGH +25, etc.) and renders SARIF/JSON/Markdown output.
• Pipeline orchestration occurs in src/skillspector/graph.py, where edges ensure sequential execution from resolve_input through report.

Frequently Asked Questions

What is the confidence threshold for LLM-validated findings in SkillSpector?

SkillSpector requires a minimum confidence score of 0.6 for findings to pass the meta-analysis stage. The apply_filter method in LLMMetaAnalyzer discards any finding where the LLM returns is_vulnerability=False or a confidence value below this threshold, effectively reducing false positives from the static analysis phase.

Can I run SkillSpector without an LLM API connection?

Yes. Pass the --no-llm flag via CLI or set use_llm=False in the SkillspectorState when invoking programmatically. In this mode, the meta_analyzer node skips LLM calls and returns the raw static findings through a fallback wrapper, allowing offline operation while sacrificing the false-positive filtering and enrichment capabilities.

How does SkillSpector handle different input types (URLs, zip files, directories)?

The resolve_input node in skillspector/nodes/resolve_input.py normalizes all input types into a local directory structure. It handles Git URLs by cloning repositories, extracts zip archives to temporary directories, and validates local paths, populating state["skill_path"] with a consistent filesystem reference for downstream analyzers.

What determines the final risk score in SkillSpector reports?

The risk calculation (implemented in skillspector/nodes/report.py) sums severity weights—CRITICAL (+50), HIGH (+25), MEDIUM (+10), and LOW (+5)—across all filtered findings. If the skill contains executable scripts (shell files, binaries), the total score is multiplied by 1.3 to reflect increased deployment risk.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}