How SkillSpector Assigns Severity Levels Based on Risk Scores

SkillSpector calculates a weighted numeric risk score from individual finding severities, applies a 1.3× multiplier if executable scripts are detected, and maps the final 0–100 score to severity bands that determine the installation recommendation.

NVIDIA's SkillSpector evaluates AI skills for security risks by first categorizing individual findings with discrete severity levels, then aggregating them into an overall risk score. This score drives the final severity band and recommendation shown in the security report.

Finding-Level Severity Classification

Every security analyzer in SkillSpector produces a Finding object with a severity field that maps to the Severity enum defined in src/skillspector/models.py. The four possible values establish the baseline for all downstream risk calculations.

class Severity(StrEnum):
    LOW = "LOW"
    MEDIUM = "MEDIUM"
    HIGH = "HIGH"
    CRITICAL = "CRITICAL"

Each finding inherits this classification during analysis, creating the raw data used to compute the skill's aggregate risk posture.

Risk Score Calculation Algorithm

The _compute_risk_score function in src/skillspector/nodes/report.py aggregates finding severities into a numeric score using fixed weights. The algorithm applies CRITICAL findings a weight of 50, HIGH findings 25, MEDIUM findings 10, and LOW findings 5.

def _compute_risk_score(findings, has_executable_scripts):
    score = 0
    for f in findings:
        sev = (f.severity or "LOW").upper()
        if sev == "CRITICAL":
            score += 50
        elif sev == "HIGH":
            score += 25
        elif sev == "MEDIUM":
            score += 10
        elif sev == "LOW":
            score += 5
    if has_executable_scripts:
        score = int(score * 1.3)
    score = min(100, max(0, score))

When the skill contains executable scripts, the function boosts the total by 1.3× to account for elevated execution risk. The final result is clamped to a 0–100 range to ensure consistent severity band mapping.

Mapping Risk Scores to Severity Bands

After calculation, the numeric score maps to a risk-severity band through threshold comparison. The _RISK_SEVERITY_BANDS tuple in src/skillspector/nodes/report.py defines the boundaries: scores 81 and above become CRITICAL, 51–80 become HIGH, 21–50 become MEDIUM, and 0–20 become LOW.

_RISK_SEVERITY_BANDS = [(81, "CRITICAL"), (51, "HIGH"), (21, "MEDIUM"), (0, "LOW")]

for threshold, band in _RISK_SEVERITY_BANDS:
    if score >= threshold:
        severity_band = band
        break

This descending threshold approach ensures the highest applicable severity band is selected immediately when the score meets or exceeds the boundary.

Installation Recommendations by Severity

The final severity band maps directly to installation guidance through the _RISK_RECOMMENDATION dictionary. Both HIGH and CRITICAL severity bands result in a DO_NOT_INSTALL recommendation, while MEDIUM yields CAUTION and LOW yields SAFE.

_RISK_RECOMMENDATION = {
    "LOW": "SAFE",
    "MEDIUM": "CAUTION",
    "HIGH": "DO_NOT_INSTALL",
    "CRITICAL": "DO_NOT_INSTALL",
}

This mapping ensures that elevated risk scores translate into actionable blocking decisions for users evaluating skill installations.

Practical Implementation Example

The following example demonstrates how individual findings translate into the final risk assessment:

from skillspector.models import Severity, Finding
from skillspector.nodes.report import _compute_risk_score

# Example findings with mixed severity levels

findings = [
    Finding(rule_id="C1", message="Critical issue", severity="CRITICAL", confidence=0.9),
    Finding(rule_id="H1", message="High issue", severity="HIGH", confidence=0.8),
    Finding(rule_id="L1", message="Low issue", severity="LOW", confidence=0.7),
]

# Calculate with executable script multiplier enabled

score, risk_severity, recommendation = _compute_risk_score(
    findings, has_executable_scripts=True
)

print(f"Risk Score: {score}")              # Output: 100 (capped from 104)

print(f"Severity Band: {risk_severity}")     # Output: CRITICAL

print(f"Recommendation: {recommendation}")   # Output: DO_NOT_INSTALL

In this scenario, the base calculation sums to 80 (50 + 25 + 5), multiplies by 1.3 to reach 104, clamps to 100, and ultimately maps to the CRITICAL band due to exceeding the 81-point threshold.

Summary

• Individual findings are classified using the Severity enum in src/skillspector/models.py with four levels: LOW, MEDIUM, HIGH, and CRITICAL.
• Risk scores are calculated in src/skillspector/nodes/report.py using weighted aggregation (CRITICAL=50, HIGH=25, MEDIUM=10, LOW=5) with a 1.3× multiplier for executable scripts.
• Severity bands are determined by threshold comparison against _RISK_SEVERITY_BANDS, where scores ≥81 become CRITICAL, ≥51 become HIGH, ≥21 become MEDIUM, and the remainder become LOW.
• Final recommendations map HIGH and CRITICAL severity to DO_NOT_INSTALL, MEDIUM to CAUTION, and LOW to SAFE.

Frequently Asked Questions

How does SkillSpector handle multiple findings of the same severity level?

SkillSpector sums the weighted values for every finding regardless of duplication. For example, two CRITICAL findings contribute 100 points to the base score (50 × 2), which when combined with other findings can quickly elevate the risk score into the CRITICAL band (≥81).

What happens if a skill contains executable scripts?

When has_executable_scripts is True, the _compute_risk_score function applies a 1.3× multiplier to the aggregated score before clamping it to the 0–100 range. This adjustment reflects the increased danger of code execution capabilities combined with identified security findings.

Why do HIGH and CRITICAL severity bands share the same recommendation?

Both HIGH and CRITICAL severity levels map to DO_NOT_INSTALL in the _RISK_RECOMMENDATION dictionary because the security risks associated with scores above 51 are deemed unacceptable for safe deployment. The distinction between HIGH and CRITICAL primarily serves to communicate the intensity of the risk rather than suggest different mitigation strategies.

Where can I find the unit tests for severity band validation?

The test suite in tests/nodes/test_report.py validates the mapping logic between findings and risk severity, including assertions that verify scores ≥51 correctly map to the HIGH severity band and that the recommendation logic returns expected values for each risk category.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}