ai-Memory Observation Limits: File Size, Character Counts, and Pagination Bounds Explained

ai-memory enforces strict content limits on observations: a 16 KB maximum body size, 200–16,384 character API bounds, and pagination capped at 200 results per request.

These boundaries are hardcoded across the Rust codebase to protect the SQLite store, maintain FTS5 index performance, and prevent abusive payload submissions. This guide breaks down each limit, where it's defined in the source code, and how the enforcement logic works in practice.

Maximum Observation Body Size (16 KB)

The foundational limit is 16 KB (16,384 bytes) of raw UTF-8 data per observation body.

This constant lives in the core sanitation module:

• File: crates/ai-memory-core/src/sanitize.rs
• Constant: OBSERVATION_BODY_MAX_BYTES

When any observation passes through ai_memory_core::Sanitizer, the truncate_utf8_bytes routine automatically truncates oversized payloads to this byte boundary. This happens regardless of whether the observation originates from the HTTP API, MCP interface, or internal consolidation hooks.

// crates/ai-memory-core/src/sanitize.rs
pub const OBSERVATION_BODY_MAX_BYTES: usize = 16 * 1024; // 16,384 bytes

pub fn truncate_utf8_bytes(s: &str, max_bytes: usize) -> &str {
    // Truncation logic ensures valid UTF-8 boundaries
}

HTTP API Character Limits

The web layer adds character-based validation with three tiers of bounds. These are defined in:

• File: crates/ai-memory-web/src/routes/api.rs

Constant	Value	Purpose
SESSION_OBSERVATIONS_MIN_BODY_CHARS	200	Rejects unreasonably short observations
SESSION_OBSERVATIONS_DEFAULT_BODY_CHARS	4,000	Suggested default for UI implementations
SESSION_OBSERVATIONS_MAX_BODY_CHARS	16,384	Hard ceiling matching the 16 KB byte limit

When you POST to /workspaces/{ws}/projects/{proj}/sessions/{id}/observations, the server validates body.length against these bounds. Violations return 400 Bad Request with explicit error messages.

// crates/ai-memory-web/src/routes/api.rs
const SESSION_OBSERVATIONS_MIN_BODY_CHARS: usize = 200;
const SESSION_OBSERVATIONS_DEFAULT_BODY_CHARS: usize = 4000;
const SESSION_OBSERVATIONS_MAX_BODY_CHARS: usize = 16384;

POST /workspaces/default/projects/demo/sessions/12345/observations
Content-Type: application/json

{
  "title": "example",
  "body": "A".repeat(20000),  // Exceeds 16,384 character limit
  "kind": "user-prompt"
}

Response:

{
  "error": "observation body exceeds maximum allowed size of 16384 characters"
}

Pagination and Result Limits

Fetch operations enforce bounded result sets to prevent memory pressure and network saturation.

From the same api.rs file:

Constant	Value	Behavior
SESSION_OBSERVATIONS_DEFAULT_LIMIT	50	Results returned when no limit param specified
SESSION_OBSERVATIONS_MAX_LIMIT	200	Ceiling applied to any limit request; values above 200 are clamped

The limit query parameter accepts any integer, but the API internally constrains it to 1–200. Requesting 500 observations silently returns 200.

// crates/ai-memory-web/src/routes/api.rs
const SESSION_OBSERVATIONS_DEFAULT_LIMIT: usize = 50;
const SESSION_OBSERVATIONS_MAX_LIMIT: usize = 200;

// Clamping logic in the handler
let limit = query.limit
    .unwrap_or(SESSION_OBSERVATIONS_DEFAULT_LIMIT)
    .clamp(1, SESSION_OBSERVATIONS_MAX_LIMIT);

Enforcement Across Interfaces

The same limits propagate through all entry points:

Interface	Enforcement Location	Implementation
Core sanitization	crates/ai-memory-core/src/sanitize.rs	Sanitized::new() truncates to OBSERVATION_BODY_MAX_BYTES
HTTP API	crates/ai-memory-web/src/routes/api.rs	Character validation + pagination clamping
MCP server	crates/ai-memory-mcp/src/server.rs	Mirrors API constants for insert_observation tool
Hooks/consolidation	crates/ai-memory-hooks/src/payload.rs	Applies OBSERVATION_BODY_MAX_BYTES when building LLM prompts

Practical Code Example

Here's how truncation behaves when inserting an oversized observation through the Rust client:

use ai_memory_core::{Sanitized, ObservationKind};
use ai_memory_store::Store;

// Construct a 20,000-byte body (exceeds 16 KB limit)
let oversized_body = "x".repeat(20_000);

// Sanitizer automatically truncates to 16,384 bytes
let observation = Sanitized::new(
    ObservationKind::UserPrompt,
    "Truncated Example",
    &oversized_body,
);

// Stored observation contains only first 16 KB
store.insert_observation(observation).await?;

The stored record silently contains the truncated content—no error is raised at the core level. The HTTP API, however, rejects rather than truncates oversized submissions.

Summary

• 16 KB (OBSERVATION_BODY_MAX_BYTES) is the absolute byte ceiling for any observation body, enforced in crates/ai-memory-core/src/sanitize.rs
• 16,384 characters is the matching API ceiling (SESSION_OBSERVATIONS_MAX_BODY_CHARS) with a 200-character floor
• 200 observations is the maximum retrievable per request (SESSION_OBSERVATIONS_MAX_LIMIT), with 50 as the default
• All interfaces—HTTP, MCP, and internal hooks—share these boundaries to maintain datastore integrity

Frequently Asked Questions

What happens if I try to store a 100 KB observation through the API?

The server rejects the request with a 400 Bad Request response stating the body exceeds 16,384 characters. You must truncate or chunk your content client-side before submission.

Does the 16 KB limit apply to the title or metadata fields?

No. The OBSERVATION_BODY_MAX_BYTES constant and related character limits apply strictly to the body field. Titles and other metadata have separate, less restrictive bounds defined in the validation layer.

Can I configure these limits at runtime or compile time?

These are hardcoded constants throughout the codebase. Changing limits requires modifying the source in sanitize.rs and api.rs, then recompiling. There are no environment variable overrides or configuration file hooks exposed.

Why are there both byte and character limits?

Byte limits (OBSERVATION_BODY_MAX_BYTES) protect SQLite storage primitives and FTS5 tokenizers from overflow. Character limits (SESSION_OBSERVATIONS_MAX_BODY_CHARS) provide user-facing validation that aligns with the byte boundary for ASCII/UTF-8 while giving API consumers predictable text-length semantics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}