# How ai-memory Captures and Ignores Files: The Complete Configuration Guide

> Master ai-memory file filtering with this guide. Learn how TOML markers and glob patterns control data capture, exclusion, and anonymization in your AI projects. Configure ai-memory for optimal performance.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: how-to-guide
- Published: 2026-09-11

---

**ai-memory uses a TOML marker file containing glob patterns to filter lifecycle events through a three-stage pipeline that compiles exclusion rules and evaluates each tool call against normalized paths to decide whether to keep, drop, or anonymize the data.**

ai-memory is a Rust-based telemetry system that records AI agent interactions without blocking IO. The process for capturing and ignoring files is governed by a declarative policy engine defined in [`crates/ai-memory-hooks/src/capture_policy.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/capture_policy.rs), which transforms repository-level configuration into compiled pattern matchers that inspect every event.

## Understanding the Capture Pipeline

The system implements a pure, IO-free policy engine that processes lifecycle events through three distinct stages:

- **Marker resolution**: Parsing the `[capture]` table from a TOML configuration
- **Policy compilation**: Building a `CapturePolicy` from validated glob patterns
- **Event inspection**: Evaluating each tool call against compiled patterns to produce a `CaptureDecision`

## Step 1: Configuring the Marker File

Repositories opt into filtering by placing a marker file (e.g., [`.ai_memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/.ai_memory.toml)) at the root. The parser extracts the `CaptureConfig` struct, which currently supports only the `ignore_paths` field—a vector of glob patterns defining files or directories to exclude.

### The CaptureConfig Structure

In [`crates/ai-memory-hooks/src/capture_policy.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/capture_policy.rs), lines 23-29, the configuration structure is defined as:

```rust
#[derive(Clone, Debug, Default, Deserialize, Serialize, PartialEq, Eq)]
pub struct CaptureConfig {
    #[serde(default)]
    pub ignore_paths: Vec<String>,
}

```

Create a marker file at your repository root to activate filtering:

```toml

# .ai_memory.toml

[capture]
ignore_paths = [
    "secret/**",          # ignore everything under ./secret

    "~/private/**",       # ignore under the user's home dir

    "build/*.tmp",        # ignore temporary build files

]

```

## Step 2: Choosing a Capture Mode

When no marker file exists, the system falls back to a default behavior defined by `CaptureMode`. This enum, located at lines 36-45 in [`capture_policy.rs`](https://github.com/akitaonrails/ai-memory/blob/main/capture_policy.rs), supports two strategies:

- **Denylist** (default): Capture everything unless it matches an ignore pattern
- **Allowlist**: Capture nothing unless a marker file is present

```rust
#[derive(Clone, Copy, Debug, Default, Deserialize, Serialize, PartialEq, Eq)]
pub enum CaptureMode { #[default] Denylist, Allowlist }

```

The helper function `repository_admits_capture` (lines 59-64) implements this logic:

```rust
pub const fn repository_admits_capture(mode: CaptureMode, marker_present: bool) -> bool {
    match mode {
        CaptureMode::Denylist => true,
        CaptureMode::Allowlist => marker_present,
    }
}

```

## Step 3: Resolving the CapturePolicy

The `CapturePolicy::resolve` method transforms the parsed configuration into an executable policy. This function, spanning lines 17-42 in [`capture_policy.rs`](https://github.com/akitaonrails/ai-memory/blob/main/capture_policy.rs), accepts the `CaptureSource`, the marker's directory path, and an optional home directory for expanding `~/` patterns.

```rust
pub fn resolve(source: CaptureSource<'_>, marker_dir: &str, home_dir: Option<&str>) -> Self { … }

```

During resolution, the system validates each glob pattern, expands relative paths against the marker location, and compiles them into `CompiledPattern` instances. If patterns are invalid or exceed limits, the policy returns `Invalid`.

## Step 4: Inspecting Events and Making Decisions

When a tool event arrives, `CapturePolicy::inspect` (lines 49-90) extracts relevant metadata including tool family, candidate file paths, and call identifiers via the `extract` function.

### CaptureDisposition Logic

The method returns a `CaptureDecision` containing a `CaptureProtocol` sent to the client as `_ai_memory_capture`. The disposition follows these rules:

- **Inactive**: If `ignore_paths` is empty, return `Keep`
- **Invalid**: If the policy is invalid and tool family is `File`, return `MetadataOnly`
- **Active**: Match each candidate path against compiled patterns:
  - Match found → `Drop` (event discarded)
  - No match → `Keep`

## Path Normalization and Pattern Matching

Before evaluation, candidate paths undergo normalization via `normalize_candidate` (lines 85-99) to handle absolute paths, collapse `..` segments, and normalize separators for Windows/Posix compatibility.

The actual pattern matching uses `glob_match` (lines 139-176), which implements limited glob syntax supporting wildcards and recursive patterns (`**`).

## Handling Dropped and Metadata-Only Events

When an event is dropped, the entire payload is discarded. For `MetadataOnly` disposition, the system constructs a safe replacement using `metadata_only_body` (lines 44-62), which preserves only:

- `session_id`
- `cwd` (current working directory)
- `tool_family`
- `tool_name`
- Optional `tool_call_id`
- Encoded `CaptureProtocol`

```rust
use ai_memory_hooks::capture_policy::metadata_only_body;

let meta = metadata_only_body(
    Some("session-123"),
    Some("/repo"),
    &decision,
);

```

## Practical Implementation Example

The following Rust example demonstrates loading a configuration, resolving the policy, and inspecting a tool payload:

```rust
use ai_memory_hooks::capture_policy::{
    CapturePolicy, CaptureSource, CaptureConfig, CaptureDisposition
};
use ai_memory_core::AgentKind;
use serde_json::json;

// Simulate parsing a marker file
let cfg = CaptureConfig { 
    ignore_paths: vec!["secret/**".into()] 
};
let policy = CapturePolicy::resolve(
    CaptureSource::Parsed(&cfg),
    "/repo",          // marker directory
    None,             // no home dir expansion needed
);

// Simulated tool payload attempting to edit a sensitive file
let payload = json!({
    "tool_name": "edit_file",
    "tool_input": { "file_path": "secret/token.txt" }
});

// Evaluate for a Codex agent
let decision = policy.inspect(AgentKind::Codex, &payload, "/repo");

assert_eq!(decision.protocol().disposition(), CaptureDisposition::Drop);

```

## Summary

- ai-memory implements **capturing and ignoring files** through a marker-based configuration system using TOML files with `[capture]` tables.
- The `CaptureConfig` struct accepts `ignore_paths` as glob patterns, processed by `CapturePolicy::resolve` in [`crates/ai-memory-hooks/src/capture_policy.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/capture_policy.rs).
- Two capture modes exist: **Denylist** (default, capture unless excluded) and **Allowlist** (capture only when marker exists), implemented via `repository_admits_capture`.
- The `inspect` method evaluates tool events against normalized paths, returning `Keep`, `Drop`, or `MetadataOnly` dispositions.
- Invalid patterns or excessive entries cause the policy to become `Invalid`, triggering metadata-only capture for file operations as a safety fallback.

## Frequently Asked Questions

### How do I exclude sensitive files from ai-memory logging?

Create a [`.ai_memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/.ai_memory.toml) file in your repository root with a `[capture]` section containing `ignore_paths` entries. Use glob patterns like `"secret/**"` or `"*.env"` to specify files or directories. The system compiles these patterns during `CapturePolicy::resolve` and applies them via `match_paths` during event inspection.

### What happens if I provide an invalid glob pattern in the ignore_paths list?

If a pattern fails validation during policy resolution, or if the list contains too many entries, `CapturePolicy::resolve` returns an `Invalid` policy state. According to the logic in `CapturePolicy::inspect`, file-related events handled by an invalid policy receive `MetadataOnly` disposition, ensuring sensitive data is not captured while flagging the configuration error.

### Can I use ai-memory in allowlist mode to capture only specific repositories?

Yes. Set `CaptureMode::Allowlist` to change the default behavior so that repositories without a marker file are completely excluded from capture. When combined with the `repository_admits_capture` helper, this mode ensures the hook only activates when `CaptureConfig` is explicitly present, as implemented in lines 59-64 of [`crates/ai-memory-hooks/src/capture_policy.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/capture_policy.rs).

### What is the difference between Drop and MetadataOnly dispositions?

`Drop` completely discards the event payload with no storage, triggered when an active policy matches a path against `ignore_paths`. `MetadataOnly`, defined in lines 44-62 of the source, replaces the full payload with a minimal JSON object containing session metadata and the `CaptureProtocol`, preserving observability headers without recording sensitive tool inputs.