What Is the LLM Semantic Analysis Stage in SkillSpector?

The LLM Semantic Analysis stage in SkillSpector serves as a second-stage filter that consumes raw static-analysis findings, invokes a large language model to evaluate each finding's semantic context—determining intent, impact, and confidence—and enriches the final security graph with AI-generated explanations and remediations.

The NVIDIA SkillSpector security scanner employs a unique two-phase architecture that separates pattern detection from contextual validation. While the first stage relies on static signatures to identify potential vulnerabilities, the LLM Semantic Analysis stage in SkillSpector provides the critical intelligence layer that determines whether a flagged pattern truly represents a security risk. This stage, implemented primarily in src/skillspector/nodes/meta_analyzer.py, transforms raw signature hits into validated, context-aware findings using结构化 structured LLM outputs and rigorous filtering logic.

The Two-Stage Pipeline: From Static Patterns to Semantic Reasoning

SkillSpector deliberately decouples detection from validation to minimize false positives. Understanding this separation clarifies why the semantic stage is essential.

Stage 1: Static Pattern Scanning

Initial nodes—prefixed with static_*, behavioral_*, and mcp_*—perform regex and AST-based scans across skill source code. These generate raw Finding objects that represent syntactically suspicious patterns but lack understanding of developer intent or business context.

Stage 2: LLM-Driven Semantic Filtering

The meta_analyzer node (implemented by the LLMMetaAnalyzer class) ingests these raw findings and subjects them to semantic reasoning. According to the SkillSpector source code, this stage evaluates whether a finding constitutes a genuine vulnerability by analyzing code context, distinguishing between malicious intent and benign implementation patterns, and calculating a confidence score before enrichment.

How the LLM Semantic Analysis Stage Works

The implementation in src/skillspector/nodes/meta_analyzer.py follows a rigorous six-step pipeline that leverages the generic LLMAnalyzerBase infrastructure while applying custom business logic for security validation.

1. Intelligent Batch Creation and File Chunking

Files exceeding the model's input token budget are automatically split into overlapping line-chunks using LLMAnalyzerBase.get_batches in src/skillspector/llm_analyzer_base.py. The system optimizes API costs by grouping only files containing at least one static finding, ensuring the LLM receives relevant context for every invocation.

2. Context-Rich Prompt Engineering

The LLMMetaAnalyzer.build_prompt method constructs a detailed prompt containing:

• Skill manifest metadata (name, description, capabilities)
• Line-numbered source code from the target file
• The specific static findings detected in that file
• Explicit evaluation instructions forcing the model to assess whether each finding represents a true vulnerability, its likely intent, and its potential security impact

3. Structured Output Schema

Unlike freeform LLM calls, the analyzer declares response_schema = MetaAnalyzerResult, a Pydantic model defined in meta_analyzer.py. This schema enforces per-finding fields including pattern_id, start_line, is_vulnerability, confidence, intent, impact, explanation, and remediation. LangChain's with_structured_output automatically validates the LLM's JSON-like replies against this schema.

4. Parallel Execution via Async Batching

The LLMMetaAnalyzer.arun_batches method (inherited from LLMAnalyzerBase) executes up to 10 concurrent LLM calls by default. Each file or chunk receives the same prompt template populated with file-specific content, maximizing throughput while respecting API rate limits.

5. Granular Finding Validation and Filtering

The apply_filter method performs precise matching using a granular key consisting of (file, rule_id, start_line, end_line). Only findings where is_vulnerability is true and confidence ≥ 0.6 survive the filter. This eliminates false positives that pass static checks but fail semantic scrutiny regarding actual exploitability.

6. Fallback Mechanisms

If the use_llm flag is disabled or the LLM call fails, the system gracefully degrades to _fallback_filtered, preserving basic static remediation data without AI enrichment to ensure the pipeline remains operational.

Key Implementation Files and Classes

Understanding the source structure reveals how the semantic stage integrates with SkillSpector's broader architecture.

The LLMMetaAnalyzer class in src/skillspector/nodes/meta_analyzer.py contains the core business logic for stage two. It extends LLMAnalyzerBase from src/skillspector/llm_analyzer_base.py, which provides generic batching, chunking (chunk_file_by_lines), and asynchronous LLM invocation machinery. The Finding data model in src/skillspector/models.py stores the enriched results, while src/skillspector/cli.py orchestrates the full pipeline execution, automatically invoking the meta-analyzer after static scans complete.

Practical Examples: Using the LLM Semantic Analysis Stage

These runnable examples demonstrate how to trigger and extend the semantic filtering capabilities.

Programmatic Execution

To invoke the meta-analyzer directly after static scanning:

from skillspector.state import SkillspectorState
from skillspector.nodes.meta_analyzer import meta_analyzer

# Configure state with static findings from stage 1

state: SkillspectorState = {
    "findings": [...],  # List of Finding objects from static scans

    "file_cache": {"my_skill.py": "def handle(): pass"},
    "manifest": {"name": "my_skill", "description": "Demo skill"},
    "model_config": {"meta_analyzer": "nvidia/openai/gpt-oss-120b"},
    "use_llm": True,
}

# Execute the LLM Semantic Analysis stage

result = meta_analyzer(state)

# Access filtered, enriched findings

for finding in result["filtered_findings"]:
    print(f"Line {finding.start_line}: {finding.explanation}")
    print(f"Remediation: {finding.remediation}")

Command-Line Invocation

Run the complete pipeline including semantic analysis via the CLI:

skillspc scan \
    --skill-dir ./my_skill \
    --model meta_analyzer=nvidia/openai/gpt-oss-120b \
    --use-llm true

This command automatically triggers the meta_analyzer node after static scans complete, performing the full two-stage analysis.

Extending with Custom Semantic Analyzers

To create a semantic discovery analyzer that generates findings directly from LLM analysis rather than filtering existing ones:

1. Subclass LLMAnalyzerBase in a new file under src/skillspector/nodes/analyzers/
2. Set response_schema = LLMAnalysisResult (the default schema)
3. Override build_prompt and parse_response to implement discovery logic

Reference implementations exist in src/skillspector/nodes/analyzers/semantic_security_discovery.py and semantic_developer_intent.py, demonstrating direct LLM-driven finding generation without prior static patterns.

Summary

• The LLM Semantic Analysis stage in SkillSpector acts as a second-stage validator that filters static findings using AI reasoning about intent, impact, and confidence.
• It uses the LLMMetaAnalyzer class in src/skillspector/nodes/meta_analyzer.py to evaluate findings via structured LLM outputs defined by the MetaAnalyzerResult schema.
• Only findings with is_vulnerability=true and a confidence score ≥ 0.6 pass the filter; rejected findings are discarded.
• Surviving findings are enriched with AI-generated explanation and remediation fields, transforming raw signatures into actionable security intelligence.
• The system gracefully falls back to static remediation if LLM calls are disabled or fail, ensuring pipeline reliability.

Frequently Asked Questions

What is the confidence threshold for the LLM Semantic Analysis stage in SkillSpector?

The apply_filter method in src/skillspector/nodes/meta_analyzer.py enforces a minimum confidence score of 0.6. Findings scoring below this threshold are discarded as false positives, even if the LLM identifies them as potential vulnerabilities. This aggressive filtering minimizes noise in the final security report.

How does SkillSpector handle files too large for the LLM context window?

The LLMAnalyzerBase.get_batches method in src/skillspector/llm_analyzer_base.py automatically chunks large files into overlapping line segments using chunk_file_by_lines. Each chunk is processed separately with full context preservation, and the meta-analyzer maintains correlation through the granular key (file, rule_id, start_line, end_line) to ensure findings map back to the correct source locations.

What happens if the LLM service is unavailable during analysis?

The system implements a fallback path via _fallback_filtered in meta_analyzer.py. When use_llm is false or API calls fail, SkillSpector retains the static findings but populates them with generic remediation templates instead of AI-enriched explanations, ensuring the security scan completes without blocking on external dependencies.

Can I use the LLM Semantic Analysis stage without running static scans first?

While LLMMetaAnalyzer is optimized for filtering existing findings, you can create discovery analyzers that operate without prior static patterns by subclassing LLMAnalyzerBase and setting response_schema = LLMAnalysisResult. The files semantic_security_discovery.py and semantic_developer_intent.py demonstrate this pattern, allowing the LLM to directly generate findings from source code analysis.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}