# How AST-Based Pattern Matching Works for Embedded Scripts in Heredocs

> Learn how AST-based pattern matching in destructive_command_guard uses ast-grep to parse heredoc scripts, distinguishing code from literals for accurate destructive command detection.

- Repository: [Jeff Emanuel/destructive_command_guard](https://github.com/Dicklesworthstone/destructive_command_guard)
- Tags: deep-dive
- Published: 2026-07-22

---

**AST-based pattern matching in `destructive_command_guard` parses extracted script content into abstract syntax trees using ast-grep to distinguish actual code from string literals and comments, enabling high-confidence detection of destructive commands hidden in shell heredocs.**

The `destructive_command_guard` (dcg) repository implements a sophisticated three-tier pipeline to isolate dangerous commands embedded in shell heredocs or inline-script flags (e.g., `python -c …`). The third and final tier leverages **AST-based pattern matching** to perform structural analysis of extracted scripts, eliminating false positives that regular expressions alone cannot catch while maintaining sub-20-millisecond performance through pre-compiled patterns and strict timeout budgets.

## The Three-Tier Detection Pipeline

The detection architecture splits analysis into distinct stages, each defined in [`src/heredoc.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/heredoc.rs). This design ensures zero-allocation fast-paths for safe commands while providing deep inspection when needed.

**Tier 1: Trigger Scanning**

The `check_triggers` function uses a `RegexSet` to perform an ultra-fast scan for heredoc delimiters or inline-script indicators. This stage allocates no memory on the fast-path and immediately returns control for commands without embedded scripts.

**Tier 2: Content Extraction**

When triggers fire, `extract_content` performs bounded extraction of the heredoc body or inline-script content. This enforces limits on total size, line count, and the number of heredocs per command, plus a strict timeout to prevent denial-of-service via malicious input.

**Tier 3: AST-Based Matching**

The extracted script content passes to the `AstMatcher` in [`src/ast_matcher.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/ast_matcher.rs). This structural analyzer gives the highest-confidence verdict by parsing the code into an abstract syntax tree (AST) and matching against language-specific destructive patterns.

## Inside the AST Matcher Implementation

The core matching logic resides in the `AstMatcher` struct, which orchestrates parsing, pattern matching, and result refinement.

### Creating the Matcher

The `AstMatcher` struct maintains pre-compiled, language-specific patterns and a hard-coded timeout budget:

```rust
pub struct AstMatcher {
    /// Patterns organised by language.
    patterns: HashMap<ScriptLanguage, Vec<PrecompiledPattern>>,
    /// Timeout for matching operations.
    timeout: Duration,
}

```

During construction (`AstMatcher::new`), the system loads default destructive patterns from [`src/config.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/config.rs), calls `precompile_perl_patterns()` to handle Perl safely, and stores the compiled results via `precompile_patterns(default_patterns())`. The production timeout is set to **20 ms** (`AST_TIMEOUT_MS`) to ensure interactive shell performance.

### The Matching Process

The public API `find_matches` processes extracted content through four guarded steps:

1. **Language support validation** – Maps `ScriptLanguage` to `ast_grep_language::SupportLang`, returning `UnsupportedLanguage` for unrecognized languages.

2. **Pattern retrieval** – Fetches pre-compiled patterns from the `HashMap`; empty pattern lists return early with `Ok(Vec::new())`.

3. **Resource guards** – Aborts if input exceeds `MAX_AST_INPUT_BYTES` or if the timeout budget expires before parsing begins.

4. **Worker thread execution** – Spawns a dedicated thread via `run_ast_match_with_timeout` to run the actual AST parsing. A shared `AtomicBool` allows the parent to cancel the worker immediately upon timeout, preventing runaway parsing from blocking the shell.

Inside the worker, the code constructs the syntax tree:

```rust
let ast = AstGrep::new(code, ast_lang);
let root = ast.root();

```

The matcher walks this tree using `root.find_all(&compiled.pattern)` for each pre-compiled rule, recording matches with their source range, line number, and a truncated preview.

### Refinement and Context Analysis

Raw AST matches undergo language-specific refinement to eliminate false positives. Functions like `refine_python_match` and `refine_javascript_match` (starting around line 640 in [`src/ast_matcher.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/ast_matcher.rs)) analyze the match context to distinguish between actual destructive calls and identical strings inside comments or literals. This refinement ensures that `rm -rf /` inside a Python docstring does not trigger a block, while `os.system("rm -rf /")` in executable code does.

### Safety Mechanisms and Fallbacks

If AST parsing fails due to unsupported language syntax, parse errors, or timeout, the matcher **fails open**—allowing the command but logging diagnostics. To maintain the zero-false-negative guarantee for destructive payloads, dcg implements **fallback regex scanners** targeting the most dangerous sinks (exec-style functions, filesystem-deleting calls). These backstops catch cases where the AST matcher cannot execute fast enough or the language lacks support.

## Integration with the Heredoc Pipeline

The orchestration logic in [`src/evaluator.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/evaluator.rs) wires the tiers together:

1. **Tier 1** (`check_triggers`) identifies potential heredocs or inline scripts.
2. **Tier 2** (`extract_content`) produces `ExtractedContent { language, content, … }` structures.
3. **Tier 3** (`AstMatcher::find_matches`) receives the extracted content, selects the appropriate language grammar, and executes the AST match.

If any returned `PatternMatch` carries `Critical` or `High` severity, dcg denies the original shell command and returns a structured JSON denial with the specific rule ID, line number, and remediation suggestion.

## Why AST-Based Matching Matters

**Precision:** By parsing scripts into syntax trees, the matcher distinguishes executable code from comments, string literals, and documentation, eliminating the false positives inherent in pure regex approaches.

**Speed:** The parser executes only after Tier 2 extraction filters out non-script content, and the 20 ms worker thread timeout ensures typical scripts finish in under 5 ms.

**Extensibility:** New destructive patterns are declared as `ast-grep` rules in [`src/config.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/config.rs) (e.g., detecting `execSync` calls or `subprocess.run` invocations) without modifying the surrounding Rust logic, allowing rapid updates to detection capabilities.

## Practical Implementation Example

The following Rust example demonstrates the complete flow from trigger detection to AST matching:

```rust
use destructive_command_guard::heredoc::{check_triggers, extract_content, ExtractionLimits};
use destructive_command_guard::ast_matcher::AstMatcher;

// Tier 1: Fast trigger detection
let cmd = "python3 -c 'import os; os.system(\"rm -rf /\")'";
assert_eq!(check_triggers(cmd), TriggerResult::Triggered);

// Tier 2: Extract the inline script with resource limits
let limits = ExtractionLimits::default();
let extraction = extract_content(cmd, &limits);

if let ExtractionResult::Extracted(contents) = extraction {
    // Tier 3: Run the AST matcher on extracted Python code
    let matcher = AstMatcher::default();
    for ec in contents {
        if let Some(block) = matcher.has_blocking_match(&ec.content, ec.language) {
            println!("Blocked destructive command: {}", block.reason);
            println!("Suggestion: {}", block.suggestion);
        }
    }
}

```

## Summary

- The AST matcher in [`src/ast_matcher.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/ast_matcher.rs) provides the third tier of detection after regex triggers and content extraction in [`src/heredoc.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/heredoc.rs).
- `AstMatcher::find_matches` runs in a dedicated worker thread with a **20 ms timeout** to prevent blocking interactive shells.
- Pattern matching uses actual syntax trees via **ast-grep**, distinguishing executable code from comments and string literals through refinement functions like `refine_python_match`.
- The system fails open on parse errors or timeouts, with fallback regex scanners ensuring dangerous commands still trigger blocks.
- New patterns are configured declaratively in [`src/config.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/config.rs) and compiled at initialization without requiring changes to the core matching logic.

## Frequently Asked Questions

### What languages does the AST matcher support?

The matcher supports Python, JavaScript, TypeScript, Ruby, Bash, Go, and PHP through native ast-grep integrations. Perl requires special handling via `precompile_perl_patterns()` due to parsing complexity, while unsupported languages trigger the fallback regex scanners.

### How does the matcher handle timeouts?

The implementation spawns a dedicated worker thread for each AST parsing operation and sets a shared `AtomicBool` flag. If parsing exceeds the 20 ms budget, the parent thread sets this flag, causing the worker to terminate immediately and return a timeout error, ensuring the shell remains responsive.

### Can the AST matcher detect destructive commands inside string literals?

No, by design the refinement phase filters out matches occurring within string literals or comments. For example, `rm -rf /` inside a Python docstring is ignored, while `os.system("rm -rf /")` in executable code is flagged. This precision eliminates false positives from documentation or data strings.

### Where are the destructive patterns defined?

Destructive patterns are declared in [`src/config.rs`](https://github.com/Dicklesworthstone/destructive_command_guard/blob/main/src/config.rs) as declarative rules that map to `ast-grep` patterns. During `AstMatcher` initialization, these rules are pre-compiled into `PrecompiledPattern` structs and organized by language in a `HashMap<ScriptLanguage, Vec<PrecompiledPattern>>`, enabling O(1) lookup during matching operations.