How SkillSpector's Two-Stage Analysis Pipeline Improves Vulnerability Detection

SkillSpector improves vulnerability detection by combining a fast static analysis stage that maximizes recall with an optional LLM-powered semantic analysis stage that filters false positives and enriches findings with intent classification and remediation guidance.

NVIDIA's open-source SkillSpector tool implements a hybrid security scanning approach that addresses the trade-off between speed and accuracy in vulnerability detection. By separating analysis into distinct static and semantic phases, the tool can rapidly identify potential issues while using contextual reasoning to eliminate false positives. This architecture is defined in the repository's workflow graph and analyzer nodes, delivering approximately 87% precision according to the project documentation.

Stage 1: Fast Static Analysis

The first stage runs automatically for every scan and focuses on high-recall detection of known vulnerability patterns. According to the README, this "Fast static analysis" phase examines skill files in a single pass to catch as many potential issues as possible without requiring external API calls.

Pattern-Matching Analyzers

SkillSpector employs a suite of 11 regex-based detectors located in files like static_patterns_anti_refusal.py and semantic_security_discovery.py. These analyzers search for known risky constructs including prompt-injection commands, unsafe system calls, and suspicious YARA signatures. Because these checks rely on regular expressions, they execute quickly but may flag patterns that are benign in specific contexts.

AST-Based Behavioral Checks

The AST analyzers—implemented in files such as semantic_developer_intent.py and behavioral_ast.py—parse Python source files to identify dangerous language constructs. These tools specifically flag nodes containing exec(), eval(), subprocess.run(), and dynamic imports that could indicate code injection or arbitrary execution vulnerabilities.

Supply-Chain Vulnerability Lookups

The static stage includes the osv_client.py module, which queries OSV.dev for known CVEs in declared dependencies. When network connectivity is unavailable, the client falls back to an offline vulnerability list, ensuring that supply-chain risks are identified even in air-gapped environments.

Stage 2: LLM Semantic Analysis

After static findings are collected, the pipeline optionally enters a second stage that uses a large language model to validate and enrich results. This stage is controlled by the meta_analyzer.py node and can be skipped using the --no-llm flag for faster scans.

Validation and False-Positive Filtering

The meta-analyzer sends each file along with its static findings to an LLM, prompting it to validate or reject each detected pattern. Because the model evaluates the complete file context, it can distinguish between dangerous code and legitimate usage that happens to match vulnerability patterns. This contextual filtering eliminates false positives that rigid regex rules cannot assess, directly contributing to the reported 87% precision rate.

Intent Classification and Remediation

For validated findings, the LLM assigns intent categories—classifying issues as malicious, negligent, or benign—and rates their impact from critical to low. The response schema, defined in MetaAnalyzerFinding and OverallAssessment structures (lines 52-96 in meta_analyzer.py), ensures the model returns actionable remediation steps and human-readable explanations of each vulnerability's risk.

Pipeline Architecture and Data Flow

The workflow graph in src/skillspector/graph.py orchestrates the two-stage process. The pipeline executes resolve_input → build_context, then dispatches all static analyzers (referenced via ANALYZER_NODE_IDS) in parallel. Their outputs flow into the meta_analyzer node, and finally into the report node that aggregates filtered findings and produces the final risk score (lines 34-52).

This ordering guarantees that every static finding undergoes LLM evaluation before the final report is emitted, creating a sequential validation chain that improves result quality without sacrificing the speed of initial detection.

Benefits of the Two-Stage Design

The separation of concerns between static and semantic analysis delivers specific advantages:

• Coverage: Static analyzers detect known patterns, AST-dangerous calls, and vulnerable dependencies, while the LLM stage adds semantic reasoning for contextual misuse not expressible by regexes.
• Speed: Static analysis runs in seconds on typical skill packages, while the LLM stage remains optional for quick scans.
• False-Positive Reduction: The static stage reports all matches, but the LLM filters out harmless matches based on surrounding code context.
• Explainability: Static findings provide pattern IDs and locations, while the LLM supplies human-readable explanations and remediation steps.
• Risk Scoring: Raw counts translate into base scores that the LLM adjusts based on intent and impact, improving final recommendations.

Together, these stages provide a high-recall first pass and a high-precision second pass, delivering more trustworthy vulnerability detections than single-stage approaches.

Usage Examples

Run a fast static-only scan without LLM validation:

skillspector scan ./my-skill/ --no-llm

Execute the full two-stage pipeline with semantic analysis:

skillspector scan ./my-skill/

Toggle the second stage programmatically using the Python API:

from skillspector import graph

# Fast static scan only

result_static = graph.invoke({
    "input_path": "./my-skill/",
    "use_llm": False,
    "output_format": "json",
})

# Full two-stage scan with LLM validation

result_full = graph.invoke({
    "input_path": "./my-skill/",
    "use_llm": True,
    "output_format": "json",
})

Summary

• SkillSpector's two-stage analysis pipeline separates fast static detection from contextual LLM validation to maximize both coverage and accuracy.
• Stage 1 uses regex patterns, AST parsing, and OSV database queries in files like static_patterns_anti_refusal.py and osv_client.py to achieve high recall.
• Stage 2 leverages meta_analyzer.py to filter false positives, classify intent, and generate remediation guidance, achieving approximately 87% precision.
• The workflow graph in graph.py ensures static findings flow through the LLM validator before final reporting, creating a sequential quality gate.
• Users can skip the LLM stage with --no-llm for rapid scans or enable it for comprehensive security assessments.

Frequently Asked Questions

What is the performance impact of running both stages?

The static analysis stage completes in seconds on typical skill packages because it processes files locally using regex and AST parsing. The LLM stage adds latency proportional to the number of findings and model response time, but it can be disabled with --no-llm when speed is prioritized over precision.

How does the pipeline handle false positives?

Stage 1 generates high-recall results that include potential false positives from pattern matching. Stage 2 addresses this by sending findings to the LLM, which evaluates the full file context to determine whether a flagged pattern is actually exploitable. This two-stage filtering is what enables the reported 87% precision rate.

Can I run SkillSpector without internet access?

Yes. The static analysis stage works entirely offline, including the AST-based checks and pattern matching. The OSV client includes a fallback to an offline vulnerability list when the network is unavailable. However, the LLM semantic analysis stage requires API connectivity to function.

What types of vulnerabilities does Stage 1 detect specifically?

Stage 1 detects prompt-injection attempts, unsafe system calls, use of dangerous functions like exec() and eval(), suspicious YARA signatures, and known CVEs in dependencies. These checks are implemented across 11 regex-based analyzers and multiple AST traversal modules in the src/skillspector/nodes/analyzers/ directory.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}