# How SkillSpector Uses LLM Semantic Analysis in Its Second Stage

> Discover how SkillSpector's second stage leverages LLM semantic analysis to filter, enrich, and validate vulnerabilities with high confidence, ensuring accurate security findings.

- Repository: [NVIDIA Corporation/SkillSpector](https://github.com/NVIDIA/SkillSpector)
- Tags: internals
- Published: 2026-06-14

---

**SkillSpector’s second stage employs a meta-analyzer that filters and enriches static findings by calling an LLM with semantic reasoning prompts, validating structured outputs against a Pydantic schema, and retaining only high-confidence vulnerabilities.**

The NVIDIA/SkillSpector repository implements a two-stage security analysis pipeline for AI skills. After static-pattern scanners produce raw findings in the first stage, the system applies **LLM semantic analysis** to evaluate intent, impact, and confidence before finalizing results. This semantic second stage transforms simple pattern matches into contextually aware security assessments.

## The Two-Stage Analysis Pipeline

SkillSpector processes skill source code through distinct phases:

1. **First stage – static-pattern scanning** – Nodes like `static_*`, `behavioral_*`, and `mcp_*` scan code and generate raw findings based on predefined signatures.
2. **Second stage – LLM-driven semantic filtering** – The `meta_analyzer` node consumes these findings, invokes an LLM per file, and applies **semantic reasoning** to determine true vulnerability status, intent, and remediation strategies.

The second stage relies on the `LLMAnalyzerBase` infrastructure but extends it with custom prompts and structured output schemas in [`src/skillspector/nodes/meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/meta_analyzer.py).

## How the Second Stage Processes Findings

The meta-analyzer orchestrates six distinct steps to transform raw static signatures into semantically validated security findings.

### Batch Creation and File Chunking

The process begins in `LLMAnalyzerBase.get_batches` (defined in [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py)). The system groups files containing at least one static finding. For files exceeding the model’s input token budget, `chunk_file_by_lines` splits the content into overlapping line-chunks to ensure complete coverage without truncation.

### Prompt Composition for Semantic Reasoning

The `LLMMetaAnalyzer.build_prompt` method (in [`src/skillspector/nodes/meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/meta_analyzer.py)) constructs a detailed prompt containing:

- **Skill manifest metadata** (name, description)
- **Line-numbered source text** of the target file
- **Static findings** discovered in that file
- **Explicit instructions** forcing semantic evaluation (e.g., "Is this a true vulnerability? What is the likely intent? What is the impact?")

This prompt design compels the LLM to perform contextual reasoning rather than pattern matching.

### Structured Response Validation

The analyzer declares `response_schema = MetaAnalyzerResult`, a Pydantic model defined in [`meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/meta_analyzer.py). This schema enforces per-finding fields including:

- `pattern_id`
- `start_line`
- `is_vulnerability` (boolean)
- `confidence` (float)
- `intent`, `impact`, `explanation`, and `remediation`

LangChain’s `with_structured_output` automatically validates the LLM’s JSON-like response against this schema, ensuring type-safe data extraction.

### Parallel Execution and Filtering

`LLMMetaAnalyzer.arun_batches` executes all per-file (or per-chunk) calls concurrently with a default maximum of 10 simultaneous requests. Each call receives the same prompt template populated with file-specific content.

After receiving LLM responses, `LLMMetaAnalyzer.apply_filter` matches outputs to original static findings using a **granular key** comprising `(file, rule_id, start_line, end_line)`. The system retains only findings where `is_vulnerability` equals `true` **and** `confidence >= 0.6`. Surviving `Finding` objects receive enrichment via the LLM-generated `explanation` and `remediation` fields.

### Fallback Path

When `use_llm` is disabled or LLM calls fail, the system invokes `_fallback_filtered` (in [`meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/meta_analyzer.py)), returning static-only remediation without semantic enrichment.

## Code Implementation Details

The second stage implementation spans several critical files:

| File | Role |
|------|------|
| [`src/skillspector/nodes/meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/meta_analyzer.py) | Implements **LLMMetaAnalyzer** class with `build_prompt`, `apply_filter`, and `MetaAnalyzerResult` schema |
| [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py) | Provides `get_batches`, `arun_batches`, and chunking logic via `LLMAnalyzerBase` |
| [`src/skillspector/models.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/models.py) | Defines the `Finding` dataclass that gets enriched with semantic metadata |
| [`src/skillspector/llm_utils.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_utils.py) | Handles LLM credential resolution and LangChain model construction |

## Practical Usage Examples

### Programmatic API Integration

Invoke the second stage directly after static analysis:

```python
from skillspector.state import SkillspectorState
from skillspector.nodes.meta_analyzer import meta_analyzer

# State populated from previous static scans

state: SkillspectorState = {
    "findings": [...],  # List of Finding objects from stage 1

    "file_cache": {"my_skill.py": "..."},
    "manifest": {"name": "my_skill", "description": "Demo"},
    "model_config": {"meta_analyzer": "nvidia/openai/gpt-oss-120b"},
    "use_llm": True,
}

# Execute semantic filtering (stage 2)

result = meta_analyzer(state)

# Access filtered and enriched findings

for finding in result["filtered_findings"]:
    print(f"{finding.explanation} | Remediation: {finding.remediation}")

```

### CLI Invocation

Run the complete pipeline including the second stage via command line:

```bash
skillspc scan \
    --skill-dir ./my_skill \
    --model meta_analyzer=nvidia/openai/gpt-oss-120b \
    --use-llm true

```

The CLI automatically triggers `meta_analyzer` after static scans complete, performing the LLM semantic analysis without additional configuration.

### Extending with Custom Semantic Analyzers

To create a semantic discovery analyzer (e.g., `semantic_security_discovery`), subclass `LLMAnalyzerBase` and set `response_schema = LLMAnalysisResult`. Override `build_prompt` and `parse_response` when implementing custom semantic logic, following the meta-analyzer’s architecture as a reference implementation.

## Summary

- **SkillSpector’s second stage** uses `LLMMetaAnalyzer` to apply LLM semantic analysis to static findings.
- **Batching and chunking** occur via `LLMAnalyzerBase.get_batches` to handle large files within token limits.
- **Semantic prompts** force the LLM to evaluate intent, impact, and confidence rather than relying on static patterns.
- **Structured validation** uses the `MetaAnalyzerResult` Pydantic schema to ensure consistent, type-safe outputs.
- **Filtering logic** retains only findings with `is_vulnerability=true` and `confidence >= 0.6`, matched by granular file coordinates.
- **Fallback mechanisms** ensure pipeline continuity when LLM calls are disabled or unavailable.

## Frequently Asked Questions

### What is the confidence threshold for keeping LLM-filtered findings?

SkillSpector retains only findings where the LLM assigns a confidence score of **0.6 or higher** (60%). This threshold is enforced in `LLMMetaAnalyzer.apply_filter` within [`meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/meta_analyzer.py).

### How does SkillSpector handle files that exceed the LLM token budget?

The system automatically splits oversized files using `chunk_file_by_lines` in `LLMAnalyzerBase.get_batches`. These overlapping line-chunks ensure the LLM receives complete context without truncation, processing each chunk separately while maintaining semantic continuity.

### Can the second stage operate without LLM calls?

Yes. If `use_llm` is set to `false` or the LLM service fails, the pipeline falls back to `_fallback_filtered` in [`meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/meta_analyzer.py), returning static findings without semantic enrichment. This ensures analysis completion even during LLM outages.

### What structured output format does the LLM return in the second stage?

The LLM returns data validated against the `MetaAnalyzerResult` Pydantic model, which includes per-finding fields: `pattern_id`, `start_line`, `is_vulnerability`, `confidence`, `intent`, `impact`, `explanation`, and `remediation`. LangChain’s `with_structured_output` validates these fields automatically.