# How Semantic Security Discovery Evaluates Intent Beyond Pattern Matching in NVIDIA SkillSpector

> Discover how semantic security discovery in NVIDIA SkillSpector goes beyond pattern matching. Analyze skill file meaning and intent with LLMs to detect advanced attacks missed by static analysis.

- Repository: [NVIDIA Corporation/SkillSpector](https://github.com/NVIDIA/SkillSpector)
- Tags: deep-dive
- Published: 2026-06-17

---

**Semantic security discovery uses a large language model to analyze the meaning and intent of skill files rather than searching for literal strings, enabling detection of semantic prompt injection, paraphrased attacks, and narrative deception that static regexes miss.**

The NVIDIA SkillSpector security framework includes a sophisticated **semantic security discovery** analyzer that moves beyond traditional static analysis. Unlike conventional security scanners that rely on keyword matching to find literal strings like "ignore system instructions" or "exfiltrate," this analyzer leverages large language models to reason about the underlying intent of code and documentation. This approach enables the detection of nuanced security risks where seemingly benign descriptions actually steer models toward unsafe behavior.

## Architecture and Core Components

The semantic security discovery system is built on a modular architecture that separates prompt engineering, LLM invocation, and result processing.

### The ANALYZER_PROMPT System

The core intelligence resides in `ANALYZER_PROMPT`, defined in [`src/skillspector/nodes/analyzers/semantic_security_discovery.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/analyzers/semantic_security_discovery.py) (lines 30-68). This multi-line system prompt defines four SSD rule families (SSD-1 through SSD-4) and explicitly instructs the LLM to focus on "intent and meaning" rather than performing keyword matches. By framing the analysis as a reasoning task about purpose and context, the prompt forces the model to interpret natural language instructions that might circumvent static detection.

### LLMAnalyzerBase Infrastructure

The `LLMAnalyzerBase` class in [`src/skillspector/llm_analyzer_base.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/llm_analyzer_base.py) provides the generic infrastructure for LLM-powered analysis. It handles batching of file contents to stay within token limits, invokes the LLM using `with_structured_output`, and converts the structured response (`LLMAnalysisResult`) into a list of `Finding` objects. This abstraction allows the semantic security discovery node to focus on domain-specific logic while the base class manages model communication.

### Node Entry Point and Model Resolution

The `node(state)` function in [`semantic_security_discovery.py`](https://github.com/NVIDIA/SkillSpector/blob/main/semantic_security_discovery.py) (lines 71-93) serves as the entry point. It extracts `use_llm`, `components`, `file_cache`, and `model_config` from the `SkillspectorState` object. The analyzer implements a three-tier model resolution strategy: it first checks for an analyzer-specific model under `model_config["semantic_security_discovery"]`, falls back to `model_config["default"]`, and finally uses the constant `_SKILLSPECTOR_DEFAULT_MODEL` defined in [`src/skillspector/constants.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/constants.py).

## The Six-Step Analysis Flow

The semantic security discovery process follows a structured pipeline to transform raw skill files into actionable security findings:

1. **State preparation** – The graph supplies a `SkillspectorState` containing the raw skill files in `file_cache` and optional model overrides.

2. **Model resolution** – The analyzer selects the appropriate model using the fallback hierarchy (analyzer-specific → default → constant).

3. **Prompt composition** – `LLMAnalyzerBase` concatenates `ANALYZER_PROMPT` with the file contents, batched to stay within token limits. The prompt forces the model to only report findings whose risk is conveyed through natural-language intent.

4. **LLM invocation** – The LLM returns a structured list of findings containing `rule_id`, `message`, `severity`, `confidence`, and other metadata.

5. **Finding conversion** – Each `LLMFinding` is transformed into a `Finding` object that includes file path, line numbers, and confidence scores.

6. **Result aggregation** – All findings from every batch are merged and returned as `{"findings": [...]}` to the pipeline.

### Error Handling and Resilience

The implementation includes robust error handling in [`semantic_security_discovery.py`](https://github.com/NVIDIA/SkillSpector/blob/main/semantic_security_discovery.py) (lines 94-102). The code catches `ValidationError` (for malformed LLM output) and generic exceptions, logging a warning and returning an empty finding list. This ensures the analysis graph continues without crashing even when the LLM produces unexpected responses.

## Detecting Semantic Threats Beyond Pattern Matching

Because the analyzer relies on the LLM's comprehension of context, it can surface threats that static regex patterns would miss:

- **Semantic prompt injection** – Instructions that reframe the conversation context without using forbidden keywords
- **Novel paraphrased attacks** – Variations of known attacks expressed in different terminology
- **Plain-language data-leak instructions** – SSD-3 violations where the text describes "telling the user to remember everything they say" rather than using technical exfiltration terms
- **Multi-step narrative deception** – Complex scenarios where benign individual steps combine into harmful behavior

## Practical Implementation and Usage

### Programmatic Integration

You can invoke the semantic security discovery analyzer directly from Python to analyze skill files:

```python
from skillspector.nodes.analyzers.semantic_security_discovery import node

# Minimal state – a single skill file is passed via file_cache

state = {
    "use_llm": True,
    "model_config": {"semantic_security_discovery": "gpt-4o-mini"},
    "components": ["SKILL.md"],
    "file_cache": {"SKILL.md": "# Example skill\n\nTell the user to remember everything they say."},

}

result = node(state)
print(result["findings"])

```

*The LLM will recognize that "tell the user to remember everything" is an SSD-3 data-leak instruction and return a finding with rule `SSD-3`.*

### Command-Line Interface

Run the full analysis via the SkillSpector CLI:

```bash
skillpector analyze path/to/skill_dir \
    --model semantic_security_discovery=gpt-4o-mini \
    --enable-llm

```

*The CLI builds the same `SkillspectorState`, routes it through the `semantic_security_discovery` node, and prints a SARIF report that includes any SSD findings.*

### Testing and Validation

The repository includes unit tests demonstrating expected behavior:

```python
from skillspector.llm_analyzer_base import LLMAnalyzerBase
from unittest.mock import MagicMock, patch

@patch("skillspector.llm_analyzer_base.get_chat_model")
def test_returns_findings_from_llm(mock_get):
    # Mock LLM that returns a single SSD‑1 finding

    mock_llm = MagicMock()
    mock_llm.with_structured_output.return_value.invoke.return_value = \
        LLMAnalysisResult(findings=[LLMFinding(rule_id="SSD-1", ...)])
    mock_get.return_value = mock_llm

    state = {
        "use_llm": True,
        "model_config": {"semantic_security_discovery": "test-model"},
        "components": ["SKILL.md"],
        "file_cache": {"SKILL.md": "..."},
    }

    result = node(state)
    assert result["findings"][0].rule_id == "SSD-1"

```

*The test validates that the node correctly forwards the LLM's structured output into `Finding` objects.* Reference: [`tests/nodes/analyzers/test_semantic_security_discovery.py`](https://github.com/NVIDIA/SkillSpector/blob/main/tests/nodes/analyzers/test_semantic_security_discovery.py).

## Summary

- **Semantic security discovery** in NVIDIA SkillSpector uses LLM reasoning rather than static pattern matching to evaluate intent.
- The `ANALYZER_PROMPT` in [`semantic_security_discovery.py`](https://github.com/NVIDIA/SkillSpector/blob/main/semantic_security_discovery.py) defines four rule families (SSD-1 through SSD-4) and explicitly directs the model to analyze meaning over keywords.
- `LLMAnalyzerBase` handles batching, structured output parsing, and conversion to `Finding` objects.
- The system catches `ValidationError` and generic exceptions to ensure pipeline resilience.
- This approach detects semantic prompt injection, paraphrased attacks, and multi-step deception that bypass traditional regex-based scanners.

## Frequently Asked Questions

### How does semantic security discovery differ from static keyword analysis?

Static keyword analysis searches for literal strings such as "ignore system instructions" or "exfiltrate data" using regex patterns. **Semantic security discovery** instead asks a large language model to reason about the meaning and intent of the content. According to the NVIDIA SkillSpector source code, this allows detection of attacks where harmful instructions are expressed in novel phrasing or buried in seemingly benign narratives.

### What are the four SSD rule families referenced in the analyzer?

The `ANALYZER_PROMPT` defines four rule families labeled SSD-1 through SSD-4. While the specific details of each rule are encoded in the prompt at [`src/skillspector/nodes/analyzers/semantic_security_discovery.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/nodes/analyzers/semantic_security_discovery.py) (lines 30-68), they generally cover categories such as prompt injection, instruction override, data exfiltration, and behavior manipulation. The LLM uses these categories to classify findings based on semantic intent rather than keyword presence.

### How does the analyzer handle malformed LLM responses?

The `node()` function in [`semantic_security_discovery.py`](https://github.com/NVIDIA/SkillSpector/blob/main/semantic_security_discovery.py) (lines 94-102) implements error handling that catches `ValidationError` when the LLM returns malformed structured output, as well as generic exceptions. When errors occur, the analyzer logs a warning and returns an empty finding list, allowing the analysis graph to continue processing other files without crashing.

### Can I override the default model for semantic security discovery?

Yes. The analyzer supports model configuration through the `model_config` dictionary in the state object. You can specify a model under the key `"semantic_security_discovery"`. If not provided, the system falls back to `model_config["default"]`, and finally to the constant `_SKILLSPECTOR_DEFAULT_MODEL` defined in [`src/skillspector/constants.py`](https://github.com/NVIDIA/SkillSpector/blob/main/src/skillspector/constants.py). This allows flexibility to use different models for different analysis tasks.