How ai-memory Captures and Ignores Files: The Complete Configuration Guide

ai-memory uses a TOML marker file containing glob patterns to filter lifecycle events through a three-stage pipeline that compiles exclusion rules and evaluates each tool call against normalized paths to decide whether to keep, drop, or anonymize the data.

ai-memory is a Rust-based telemetry system that records AI agent interactions without blocking IO. The process for capturing and ignoring files is governed by a declarative policy engine defined in crates/ai-memory-hooks/src/capture_policy.rs, which transforms repository-level configuration into compiled pattern matchers that inspect every event.

Understanding the Capture Pipeline

The system implements a pure, IO-free policy engine that processes lifecycle events through three distinct stages:

• Marker resolution: Parsing the [capture] table from a TOML configuration
• Policy compilation: Building a CapturePolicy from validated glob patterns
• Event inspection: Evaluating each tool call against compiled patterns to produce a CaptureDecision

Step 1: Configuring the Marker File

Repositories opt into filtering by placing a marker file (e.g., .ai_memory.toml) at the root. The parser extracts the CaptureConfig struct, which currently supports only the ignore_paths field—a vector of glob patterns defining files or directories to exclude.

The CaptureConfig Structure

In crates/ai-memory-hooks/src/capture_policy.rs, lines 23-29, the configuration structure is defined as:

#[derive(Clone, Debug, Default, Deserialize, Serialize, PartialEq, Eq)]
pub struct CaptureConfig {
    #[serde(default)]
    pub ignore_paths: Vec<String>,
}

Create a marker file at your repository root to activate filtering:

# .ai_memory.toml

[capture]
ignore_paths = [
    "secret/**",          # ignore everything under ./secret

    "~/private/**",       # ignore under the user's home dir

    "build/*.tmp",        # ignore temporary build files

]

Step 2: Choosing a Capture Mode

When no marker file exists, the system falls back to a default behavior defined by CaptureMode. This enum, located at lines 36-45 in capture_policy.rs, supports two strategies:

• Denylist (default): Capture everything unless it matches an ignore pattern
• Allowlist: Capture nothing unless a marker file is present

#[derive(Clone, Copy, Debug, Default, Deserialize, Serialize, PartialEq, Eq)]
pub enum CaptureMode { #[default] Denylist, Allowlist }

The helper function repository_admits_capture (lines 59-64) implements this logic:

pub const fn repository_admits_capture(mode: CaptureMode, marker_present: bool) -> bool {
    match mode {
        CaptureMode::Denylist => true,
        CaptureMode::Allowlist => marker_present,
    }
}

Step 3: Resolving the CapturePolicy

The CapturePolicy::resolve method transforms the parsed configuration into an executable policy. This function, spanning lines 17-42 in capture_policy.rs, accepts the CaptureSource, the marker's directory path, and an optional home directory for expanding ~/ patterns.

pub fn resolve(source: CaptureSource<'_>, marker_dir: &str, home_dir: Option<&str>) -> Self { … }

During resolution, the system validates each glob pattern, expands relative paths against the marker location, and compiles them into CompiledPattern instances. If patterns are invalid or exceed limits, the policy returns Invalid.

Step 4: Inspecting Events and Making Decisions

When a tool event arrives, CapturePolicy::inspect (lines 49-90) extracts relevant metadata including tool family, candidate file paths, and call identifiers via the extract function.

CaptureDisposition Logic

The method returns a CaptureDecision containing a CaptureProtocol sent to the client as _ai_memory_capture. The disposition follows these rules:

• Inactive: If ignore_paths is empty, return Keep
• Invalid: If the policy is invalid and tool family is File, return MetadataOnly
• Active: Match each candidate path against compiled patterns:
  • Match found → Drop (event discarded)
  • No match → Keep

Path Normalization and Pattern Matching

Before evaluation, candidate paths undergo normalization via normalize_candidate (lines 85-99) to handle absolute paths, collapse .. segments, and normalize separators for Windows/Posix compatibility.

The actual pattern matching uses glob_match (lines 139-176), which implements limited glob syntax supporting wildcards and recursive patterns (**).

Handling Dropped and Metadata-Only Events

When an event is dropped, the entire payload is discarded. For MetadataOnly disposition, the system constructs a safe replacement using metadata_only_body (lines 44-62), which preserves only:

• session_id
• cwd (current working directory)
• tool_family
• tool_name
• Optional tool_call_id
• Encoded CaptureProtocol

use ai_memory_hooks::capture_policy::metadata_only_body;

let meta = metadata_only_body(
    Some("session-123"),
    Some("/repo"),
    &decision,
);

Practical Implementation Example

The following Rust example demonstrates loading a configuration, resolving the policy, and inspecting a tool payload:

use ai_memory_hooks::capture_policy::{
    CapturePolicy, CaptureSource, CaptureConfig, CaptureDisposition
};
use ai_memory_core::AgentKind;
use serde_json::json;

// Simulate parsing a marker file
let cfg = CaptureConfig { 
    ignore_paths: vec!["secret/**".into()] 
};
let policy = CapturePolicy::resolve(
    CaptureSource::Parsed(&cfg),
    "/repo",          // marker directory
    None,             // no home dir expansion needed
);

// Simulated tool payload attempting to edit a sensitive file
let payload = json!({
    "tool_name": "edit_file",
    "tool_input": { "file_path": "secret/token.txt" }
});

// Evaluate for a Codex agent
let decision = policy.inspect(AgentKind::Codex, &payload, "/repo");

assert_eq!(decision.protocol().disposition(), CaptureDisposition::Drop);

Summary

• ai-memory implements capturing and ignoring files through a marker-based configuration system using TOML files with [capture] tables.
• The CaptureConfig struct accepts ignore_paths as glob patterns, processed by CapturePolicy::resolve in crates/ai-memory-hooks/src/capture_policy.rs.
• Two capture modes exist: Denylist (default, capture unless excluded) and Allowlist (capture only when marker exists), implemented via repository_admits_capture.
• The inspect method evaluates tool events against normalized paths, returning Keep, Drop, or MetadataOnly dispositions.
• Invalid patterns or excessive entries cause the policy to become Invalid, triggering metadata-only capture for file operations as a safety fallback.

Frequently Asked Questions

How do I exclude sensitive files from ai-memory logging?

Create a .ai_memory.toml file in your repository root with a [capture] section containing ignore_paths entries. Use glob patterns like "secret/**" or "*.env" to specify files or directories. The system compiles these patterns during CapturePolicy::resolve and applies them via match_paths during event inspection.

What happens if I provide an invalid glob pattern in the ignore_paths list?

If a pattern fails validation during policy resolution, or if the list contains too many entries, CapturePolicy::resolve returns an Invalid policy state. According to the logic in CapturePolicy::inspect, file-related events handled by an invalid policy receive MetadataOnly disposition, ensuring sensitive data is not captured while flagging the configuration error.

Can I use ai-memory in allowlist mode to capture only specific repositories?

Yes. Set CaptureMode::Allowlist to change the default behavior so that repositories without a marker file are completely excluded from capture. When combined with the repository_admits_capture helper, this mode ensures the hook only activates when CaptureConfig is explicitly present, as implemented in lines 59-64 of crates/ai-memory-hooks/src/capture_policy.rs.

What is the difference between Drop and MetadataOnly dispositions?

Drop completely discards the event payload with no storage, triggered when an active policy matches a path against ignore_paths. MetadataOnly, defined in lines 44-62 of the source, replaces the full payload with a minimal JSON object containing session metadata and the CaptureProtocol, preserving observability headers without recording sensitive tool inputs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}