# How SkillSpector's Two-Stage Static and LLM Analysis Pipeline Works

> Discover how SkillSpector's two-stage pipeline unites static analysis and LLM validation to find AI agent skill vulnerabilities, ensuring critical findings are never missed.

- Repository: [NVIDIA Corporation/SkillSpector](https://github.com/NVIDIA/SkillSpector)
- Tags: internals
- Published: 2026-06-24

---

**SkillSpector's two-stage static and LLM analysis pipeline combines fast deterministic static analysis with contextual LLM validation to detect security vulnerabilities in AI agent skills while ensuring critical findings are never silently discarded.**

SkillSpector is an open-source security scanner for AI agent skills developed by NVIDIA. Its **two-stage static and LLM analysis pipeline** processes skill files through a LangGraph workflow that first applies signature-based detection, then uses large language models to validate findings and enrich them with contextual risk assessments and remediation guidance.

## Phase 1: Static Analysis Engine

The pipeline begins with deterministic static analysis orchestrated by the **`build_context`** node in [`src/skillspector/nodes/build_context.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/build_context.py). This node materializes skill files into a **file cache** and prepares code context for downstream analysis.

The static analysis phase operates through several specialized analyzer nodes located in `src/skillspector/nodes/analyzers/*.py`:

- **[`static_yara.py`](https://github.com/NVIDIA/SkillSpector/blob/main/static_yara.py)** – Applies YARA signatures to detect known malicious patterns
- **[`static_patterns_privilege_escalation.py`](https://github.com/NVIDIA/SkillSpector/blob/main/static_patterns_privilege_escalation.py)** – Performs pattern-based checks for privilege escalation vectors
- **[`osv_client.py`](https://github.com/NVIDIA/SkillSpector/blob/main/osv_client.py)** – Queries the Open Source Vulnerabilities database for dependency risks

Each analyzer independently processes files and returns structured **`Finding`** objects defined in [`src/skillspector/models.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/models.py). These findings capture:

- **rule_id** and **message** identifying the detection
- **severity** and **confidence** levels
- **Line numbers** and **matched text** for precise location
- Optional **remediation** guidance

The findings accumulate in the **`SkillspectorState`** typed dictionary (defined in [`src/skillspector/state.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/state.py)), which flows through the LangGraph workflow.

## Phase 2: LLM-Driven Filtering and Enrichment

After static analysis completes, the **`meta_analyzer`** node in [`src/skillspector/nodes/meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/meta_analyzer.py) executes the second stage. This phase leverages the **`LLMMetaAnalyzer`** class (a subclass of `LLMAnalyzerBase` from [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py)) to perform contextual validation.

For every file containing static findings, the pipeline constructs a per-file LLM request using the **`PER_FILE_ANALYSIS_PROMPT`** template. The prompt includes:

1. Skill metadata and manifest information
2. Complete file contents from the file cache
3. Raw static findings from phase one
4. Security-focused instructions that explicitly forbid the model from trusting self-declared "safe" statements

The LLM response must conform to the **`MetaAnalyzerResult`** Pydantic schema, ensuring structured JSON output containing:

- **`findings`** – Enriched entries with `is_vulnerability`, `confidence`, `intent`, `impact`, `explanation`, and `remediation` fields
- **`overall_assessment`** – A risk-level summary for the entire file

The **`apply_filter`** routine merges the LLM response with original static findings using a **fail-closed** strategy:

- **Preserves every HIGH or CRITICAL static finding**, adding an `"llm-unconfirmed"` tag when the LLM does not confirm the vulnerability
- **Discards or downgrades** lower-severity findings based on the LLM's contextual verdict
- Adds human-readable explanations and targeted remediation guidance to confirmed findings

## LangGraph Workflow Architecture

The complete pipeline is assembled in **[`src/skillspector/graph.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/graph.py)** as a LangGraph workflow with the following execution flow:

```

resolve_input → build_context → [static analyzers] → meta_analyzer → report

```

- **`resolve_input`** parses CLI arguments and skill manifests
- **`build_context`** creates the file cache and distributes file lists to all analyzers registered in **`ANALYZER_NODE_IDS`**
- **Static analyzers** run in parallel, emitting findings into the shared state
- **`meta_analyzer`** performs per-file LLM calls via [`src/skillspector/llm_utils.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_utils.py) (which handles credential resolution and LangChain `ChatModel` construction)
- **`report`** in [`src/skillspector/report.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/report.py) serializes final findings to SARIF or JSON format

This architecture provides **defense-in-depth**: deterministic static signatures catch obvious problems cheaply, while the LLM adds contextual reasoning and false-positive reduction.

## Fail-Closed Safety Mechanisms

SkillSpector guarantees **fail-closed** semantics to prevent security regressions. If LLM calls are disabled via `--no-llm` (setting `use_llm=False`) or if API calls fail, the pipeline activates fallback mechanisms in [`meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/meta_analyzer.py):

- **`_fallback_filtered`** applies confidence-based heuristics to static findings and adds default remediations
- **`_passthrough_with_defaults`** ensures critical alerts pass through with default metadata even during total LLM failure

These safeguards ensure the pipeline never silently drops HIGH or CRITICAL severity findings, regardless of LLM availability or configuration.

## Running the Pipeline

Execute the full pipeline from the command line with LLM validation enabled:

```bash
skillspector scan path/to/skill \
    --model meta_analyzer=gpt-4o-mini \
    --output results.sarif

```

For programmatic integration in Python applications or test suites:

```python
from skillspector.graph import graph
from skillspector.state import SkillspectorState

# Initialize state with manifest and configuration

state = SkillspectorState(
    manifest={"name": "demo-skill", "description": "example"},
    file_cache={},
    use_llm=True,
    model_config={"meta_analyzer": "gpt-4o-mini"},
)

# Execute the complete workflow

final_state = graph.invoke(state)

# Access LLM-filtered findings

for finding in final_state["filtered_findings"]:
    print(f"{finding.rule_id}: {finding.message} (confidence={finding.confidence:.2f})")

```

Run without LLM validation for CI environments lacking API credentials:

```bash
skillspector scan path/to/skill --no-llm

```

This command triggers the heuristic fallback mode, applying static analysis with default confidence scoring and remediation templates.

## Summary

- **SkillSpector's two-stage static and LLM analysis pipeline** first applies deterministic static analysis via YARA signatures, pattern matching, and OSV database queries in `src/skillspector/nodes/analyzers/`.
- The **`meta_analyzer`** node in [`src/skillspector/nodes/meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/meta_analyzer.py) validates findings using the **`LLMMetaAnalyzer`** class and the **`MetaAnalyzerResult`** schema to enrich outputs with contextual risk assessments.
- The pipeline implements **fail-closed** safety through **`_fallback_filtered`** and **`_passthrough_with_defaults`**, ensuring HIGH and CRITICAL findings are never dropped.
- The entire workflow is orchestrated through LangGraph in [`src/skillspector/graph.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/graph.py), using **`SkillspectorState`** to pass data between nodes from `resolve_input` to final `report` generation.

## Frequently Asked Questions

### What happens if the LLM API is unavailable during analysis?

SkillSpector implements fail-closed semantics in [`src/skillspector/nodes/meta_analyzer.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/meta_analyzer.py). If LLM calls fail or are disabled with `--no-llm`, the pipeline automatically invokes **`_fallback_filtered`** or **`_passthrough_with_defaults`**, which preserve all HIGH and CRITICAL static findings while applying confidence-based heuristics and default remediations.

### How does SkillSpector prevent the LLM from ignoring self-declared "safe" code?

The **`PER_FILE_ANALYSIS_PROMPT`** template includes explicit security instructions that forbid the model from trusting self-declared "safe" statements or benign intentions in code comments. This instruction set, processed by the **`LLMMetaAnalyzer`** class, ensures the LLM evaluates actual implementation behavior rather than developer assertions.

### What static analysis tools does SkillSpector use in the first stage?

The static analysis phase employs multiple specialized analyzers in `src/skillspector/nodes/analyzers/`, including **[`static_yara.py`](https://github.com/NVIDIA/SkillSpector/blob/main/static_yara.py)** for signature matching, **[`static_patterns_privilege_escalation.py`](https://github.com/NVIDIA/SkillSpector/blob/main/static_patterns_privilege_escalation.py)** for privilege escalation detection, and **[`osv_client.py`](https://github.com/NVIDIA/SkillSpector/blob/main/osv_client.py)** for dependency vulnerability scanning against the Open Source Vulnerabilities database.

### Can I customize which LLM model validates the findings?

Yes. The pipeline accepts model configuration through the `--model` CLI flag or the `model_config` dictionary in `SkillspectorState`. The **`meta_analyzer`** key specifies the model for the LLM stage (e.g., `meta_analyzer=gpt-4o-mini`), with credentials resolved via [`src/skillspector/llm_utils.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_utils.py) using standard environment variables or configuration files.