# How Shell Command Expansion Works in Sandcastle Prompts: A Deep Dive into `!`command` Syntax

> Understand shell command expansion with `!command` syntax in Sandcastle prompts. Learn how Sandcastle executes commands in containers, replaces blocks with stdout, and prevents injection.

- Repository: [Matt Pocock/sandcastle](https://github.com/mattpocock/sandcastle)
- Tags: deep-dive
- Published: 2026-05-24

---

**Sandcastle enables dynamic prompts by executing shell commands via the `!`command` syntax inside sandboxed containers, replacing the block with command stdout while using invisible markers to prevent code injection through argument substitution.**

The mattpocock/sandcastle repository implements a robust **shell command expansion** system that allows prompts to embed live data from the repository or host system. This feature uses the distinctive syntax ``!`command` `` to execute arbitrary shell commands within a sandboxed environment, automatically inserting their output into the final prompt sent to the LLM. The implementation spans several modules including [`src/PromptArgumentSubstitution.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptArgumentSubstitution.ts) and [`src/PromptPreprocessor.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptPreprocessor.ts), employing a multi-stage pipeline that prioritizes security and reproducibility.

## The Three-Step Processing Pipeline

The **shell command expansion** process operates through three distinct phases that ensure safe execution and proper substitution order.

### Step 1: Argument Substitution

First, user-supplied arguments are interpolated into the prompt template using `substitutePromptArgs` in [`src/PromptArgumentSubstitution.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptArgumentSubstitution.ts). This step must distinguish between ``!`command` `` blocks that exist in the original template versus identical-looking strings that arrive through argument values, as the latter must remain literal text and never execute.

### Step 2: Marking Shell Blocks with Invisible Markers

To prevent template injection attacks, the system identifies raw ``!`command` `` blocks in the original template and prefixes them with an invisible marker (`\x01`). In [`src/PromptArgumentSubstitution.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptArgumentSubstitution.ts), the `SHELL_BLOCK_PATTERN` regex finds these blocks and wraps them using `SHELL_BLOCK_MARKER`, creating a pattern like ``!\x01`command` `` that only the pre-processor recognizes.

### Step 3: Pre-processing and Execution

The `PromptPreprocessor.preprocessPrompt` method in [`src/PromptPreprocessor.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptPreprocessor.ts) scans for marked blocks using the regex:

```typescript
const MARKED_SHELL_BLOCK_PATTERN = new RegExp(`!${SHELL_BLOCK_MARKER}\`([^\\`]+)\\``, "g");

```

For each match, the system:

- Executes the command via `sandbox.exec` with the current working directory
- Enforces a **30-second timeout** using `withTimeout(PROMPT_EXPANSION_TIMEOUT_MS)`
- Throws `PromptExpansionTimeoutError` if the limit is exceeded
- Throws `PromptError` for non-zero exit codes, including stderr details
- Replaces the entire block with trimmed stdout on success

After processing, the invisible markers are stripped from the final output.

## Security Architecture and Safety Mechanisms

The **shell command expansion** system implements multiple defense layers to ensure safe execution within the sandbox.

### The Invisible Marker System

The **marker** (`\x01`) serves as a cryptographic boundary between template-defined commands and user-supplied data. Because [`src/PromptArgumentSubstitution.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptArgumentSubstitution.ts) only marks blocks present in the original template, any ``!`command` `` string injected through argument substitution remains unmarked and is treated as literal text by [`src/PromptPreprocessor.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptPreprocessor.ts).

### Timeout and Error Handling

Commands execute with a hard timeout of **30 seconds** (`PROMPT_EXPANSION_TIMEOUT_MS`). When exceeded, the system raises `PromptExpansionTimeoutError` containing the specific command and timeout duration. Non-zero exits trigger `PromptError` with the exit code and stderr output, preventing silent failures from contaminating the LLM context.

### Token Accounting

The pre-processor logs approximate token consumption using the formula `Math.ceil(stdout.length / 4)`, providing visibility into how command output affects context window usage.

## Implementation Examples

### Basic Shell Expansion in Prompts

Define a template with embedded commands:

```typescript
const prompt = `
Current branch: !\`git rev-parse --abbrev-ref HEAD\`
Today's date: !\`date +%Y-%m-%d\`
`;

```

During processing, `preprocessPrompt` executes these inside the sandbox and produces:

```

Current branch: main
Today's date: 2026-05-24

```

### Injection Prevention Through Argument Substitution

When user arguments contain shell-like syntax, the system preserves them as literals:

```typescript
// User-supplied injection attempt
const args = { MESSAGE: "!`rm -rf /`" };
const rawTemplate = "Show me the payload: {{MESSAGE}}";
const substituted = substitutePromptArgs(rawTemplate, args);
// Result: "Show me the payload: !`rm -rf /`"
// The block lacks the marker, so pre-processor ignores it

```

### Internals: Marking Implementation

From [`src/PromptArgumentSubstitution.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptArgumentSubstitution.ts), the marking logic uses regex substitution:

```typescript
const SHELL_BLOCK_PATTERN = /!`([^`]+)`/g;
const SHELL_BLOCK_MARKER = "\x01";

function substitutePromptArgs(template, args) {
  return template
    .replaceAll(/* argument interpolation */)
    // Mark only original template shell blocks
    .replace(SHELL_BLOCK_PATTERN, `!${SHELL_BLOCK_MARKER}\`$1\``);
}

```

### Internals: Expansion Implementation

The [`src/PromptPreprocessor.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptPreprocessor.ts) execution flow handles concurrency and error propagation:

```typescript
const matches = [...prompt.matchAll(MARKED_SHELL_BLOCK_PATTERN)];

return Effect.gen(function* () {
  const display = yield* Display;
  return yield* display.taskLog("Expanding shell expressions", (message) =>
    Effect.gen(function* () {
      const results = yield* Effect.all(
        matches.map((match) => {
          const command = match[1]!;
          return sandbox.exec(command, { cwd })
            .flatMap((res) =>
              res.exitCode !== 0
                ? Effect.fail(new PromptError({ message: `Command \`${command}\` exited with code ${res.exitCode}: ${res.stderr}` }))
                : Effect.succeed(res.stdout.trimEnd()))
            .pipe(withTimeout(PROMPT_EXPANSION_TIMEOUT_MS, () =>
              new PromptExpansionTimeoutError({ message: `Shell expression \`${command}\` timed out`, timeoutMs: PROMPT_EXPANSION_TIMEOUT_MS, expression: command })));
        }),
        { concurrency: "unbounded" }
      );
      // Replace matches in reverse order to maintain index stability
      let result = prompt;
      for (let i = matches.length - 1; i >= 0; i--) {
        const { index } = matches[i]!;
        result = result.slice(0, index) + results[i] + result.slice(index + matches[i]![0].length);
      }
      return result.replaceAll(SHELL_BLOCK_MARKER, "");
    })
  );
});

```

## Summary

- **Sandcastle** implements **shell command expansion** using the ``!`command` `` syntax wrapped in a three-stage pipeline: argument substitution, marker insertion, and pre-processing execution.
- **Security** relies on the invisible `\x01` marker inserted by [`src/PromptArgumentSubstitution.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptArgumentSubstitution.ts) to distinguish template commands from injected literal text.
- **Execution** occurs within the sandbox via `sandbox.exec` with a **30-second timeout**, throwing `PromptExpansionTimeoutError` or `PromptError` on failures.
- **Replacement** happens in [`src/PromptPreprocessor.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptPreprocessor.ts) using regex matching and reverse-index replacement to maintain string stability during multiple substitutions.
- **Token accounting** estimates usage as `Math.ceil(stdout.length / 4)` for monitoring context window consumption.

## Frequently Asked Questions

### What is the exact timeout for shell command expansion in Sandcastle?

Commands executed via **shell command expansion** have a hard timeout of **30 seconds** defined by `PROMPT_EXPANSION_TIMEOUT_MS` in [`src/PromptPreprocessor.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptPreprocessor.ts). If exceeded, the system raises `PromptExpansionTimeoutError` and halts prompt processing.

### How does Sandcastle prevent command injection through prompt arguments?

The system uses an **invisible marker** (`\x01`) inserted by `substitutePromptArgs` in [`src/PromptArgumentSubstitution.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptArgumentSubstitution.ts) only for blocks present in the original template. Arguments injected through user input lack this marker, causing `preprocessPrompt` to treat them as literal text rather than executable commands.

### What happens when a shell command returns a non-zero exit code?

When `sandbox.exec` returns a non-zero exit code, the pre-processor throws `PromptError` containing the specific exit code and stderr output. This ensures that failed commands do not silently inject error messages or empty strings into the LLM prompt.

### Can multiple shell commands be expanded in a single prompt?

Yes, [`src/PromptPreprocessor.ts`](https://github.com/mattpocock/sandcastle/blob/main/src/PromptPreprocessor.ts) processes all marked blocks concurrently using `Effect.all` with `{ concurrency: "unbounded" }`. It replaces matches in reverse order to maintain index stability, allowing unlimited shell expansions within a single template.