How the Meta Analyzer in SkillSpector Filters False Positives: A Technical Deep Dive

The meta analyzer in SkillSpector filters false positives by using a Large Language Model (LLM) to re-evaluate static findings per file, keeping only those marked as true vulnerabilities with confidence scores of 0.6 or higher while enriching them with remediation details.

The NVIDIA SkillSpector repository implements a sophisticated pipeline for analyzing code security, where the meta analyzer serves as a critical gatekeeper between raw static analysis and final vulnerability reports. This component leverages structured LLM prompts to validate findings produced by static scanners, significantly reducing noise in security audits. Understanding how this node processes and filters results helps developers tune their security workflows and interpret confidence scores effectively.

Overview of the Meta Analyzer Architecture

Positioned after static scanners in the SkillSpector graph, the meta analyzer operates on Finding objects that require secondary validation. According to the source code in src/skillspector/nodes/meta_analyzer.py, the meta_analyzer() function first checks the use_llm flag in the state—if disabled, it immediately routes to _fallback_filtered() without performing any filtering.

The primary execution path involves the LLMMetaAnalyzer class, which batches findings per file and constructs detailed prompts containing the skill's manifest, file content, and raw static findings. This design ensures the LLM has complete context to distinguish between exploitable vulnerabilities and benign code patterns.

The False Positive Filtering Workflow

1. Structured LLM Evaluation

The meta analyzer sends batched findings to an LLM using a strict Pydantic schema defined in MetaAnalyzerResult. Each potential vulnerability returns as a MetaAnalyzerFinding object with the following structure:

class MetaAnalyzerFinding(BaseModel):
    pattern_id: str
    start_line: int | None = None
    end_line: int | None = None
    is_vulnerability: bool
    confidence: float = Field(ge=0.0, le=1.0)
    intent: Literal["malicious", "neglectful", "benign"]
    impact: Literal["critical", "high", "medium", "low"]
    explanation: str = ""
    remediation: str = ""

This schema enforces validation at the API level, eliminating brittle JSON parsing while capturing critical metadata including boolean vulnerability status and numerical confidence scores.

2. Confidence-Based Filtering Logic

Inside apply_filter(), the meta analyzer implements a hard confidence threshold of 0.6. Any finding where is_vulnerability=False or confidence < 0.6 gets discarded immediately:

if not item.get("pattern_id") or not item.get("is_vulnerability"):
    continue
if float(item.get("confidence", 0.7)) < 0.6:
    continue

Only findings meeting these criteria proceed to the granular matching phase.

3. Granular Key Matching

To disambiguate multiple findings sharing the same rule ID within a single file, apply_filter() employs a tiered keying strategy:

exact_key = (f.file, f.rule_id, f.start_line, f.end_line)
start_only_key = (f.file, f.rule_id, f.start_line, None)
coarse_key = (f.file, f.rule_id)

if exact_key in confirmed_granular:
    # Enrich and keep

elif start_only_key in confirmed_granular:
    # Enrich and keep

elif coarse_key in confirmed_coarse:
    # Enrich and keep

else:
    continue  # Drop false positive

This three-tier approach—exact line matching, start-line matching, and rule-level matching—ensures the meta analyzer retains findings even when line numbers shift slightly, while still filtering out unrelated matches.

Fallback Behavior and Error Handling

When use_llm=False or when the LLM call fails, the system invokes _fallback_filtered() as implemented in src/skillspector/nodes/meta_analyzer.py. This fallback preserves all original findings but enriches them with default remediation text from get_remediation(), ensuring the pipeline continues without data loss while skipping the computationally expensive LLM evaluation.

Integration Example

The following example demonstrates direct invocation of the meta analyzer node using SkillSpector's state management:

from skillspector.nodes.meta_analyzer import meta_analyzer
from skillspector.state import SkillspectorState
from skillspector.models import Finding

state = SkillspectorState(
    findings=[
        Finding(
            rule_id="E2",
            message="Potential command injection",
            severity="high",
            confidence=0.9,
            file="skill.py",
            start_line=12,
            end_line=12,
            remediation=None,
            tags=[],
            context="os.system(user_input)",
            matched_text="os.system(user_input)",
        )
    ],
    file_cache={"skill.py": "def execute():\n    os.system(user_input)"},
    manifest={"name": "example"},
    use_llm=True,
    model_config={"meta_analyzer": "gpt-4o-mini"},
)

result = meta_analyzer(state)
filtered = result["filtered_findings"]

When use_llm=True, this returns only findings the LLM confirmed as true vulnerabilities. Setting use_llm=False would return the full list with generic remediation templates.

Summary

• The meta analyzer in SkillSpector filters false positives by submitting static findings to an LLM for secondary validation using structured Pydantic schemas.
• Only findings with is_vulnerability=True and confidence scores ≥ 0.6 survive the filtering process.
• The apply_filter() function uses granular key matching based on file path, rule ID, and line numbers to accurately merge LLM responses with original findings.
• When LLM evaluation is disabled via use_llm=False, the system falls back to _fallback_filtered(), preserving all findings but applying default remediation templates.
• Core implementation resides in src/skillspector/nodes/meta_analyzer.py, with supporting batch logic in src/skillspector/llm_analyzer_base.py.

Frequently Asked Questions

What confidence threshold does the SkillSpector meta analyzer use?

The meta analyzer enforces a minimum confidence threshold of 0.6 as implemented in the apply_filter() function. Any finding the LLM evaluates with a confidence score below this value gets filtered out as a false positive, regardless of other metadata.

How does the meta analyzer match LLM responses to specific code findings?

The system uses a three-tier key matching strategy in apply_filter(): first attempting exact matches using (file, rule_id, start_line, end_line), then falling back to (file, rule_id, start_line), and finally to (file, rule_id). This granular approach handles line number variations while preventing incorrect matches between different findings.

What happens if the LLM is unavailable or disabled in SkillSpector?

When use_llm=False or the LLM call fails, the meta analyzer invokes _fallback_filtered(), which returns all original findings unchanged except for adding default remediation text. This ensures the security pipeline continues operating without blocking on external API dependencies.

Where is the meta analyzer logic implemented in the SkillSpector codebase?

The primary implementation lives in src/skillspector/nodes/meta_analyzer.py, containing the meta_analyzer() entry point, apply_filter() matching logic, and Pydantic response models. Supporting infrastructure for batch processing and token management resides in src/skillspector/llm_analyzer_base.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}