# ai-Memory Observation Limits: File Size, Character Counts, and Pagination Bounds Explained

> Understand ai-memory observation limits including file size, character counts, and pagination bounds. Learn how to optimize your data for AI memory with this essential guide.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: deep-dive
- Published: 2026-09-01

---

**ai-memory enforces strict content limits on observations: a 16 KB maximum body size, 200–16,384 character API bounds, and pagination capped at 200 results per request.**

These boundaries are hardcoded across the Rust codebase to protect the SQLite store, maintain FTS5 index performance, and prevent abusive payload submissions. This guide breaks down each limit, where it's defined in the source code, and how the enforcement logic works in practice.

## Maximum Observation Body Size (16 KB)

The foundational limit is **16 KB** (16,384 bytes) of raw UTF-8 data per observation body.

This constant lives in the core sanitation module:

- **File:** [`crates/ai-memory-core/src/sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs)
- **Constant:** `OBSERVATION_BODY_MAX_BYTES`

When any observation passes through `ai_memory_core::Sanitizer`, the `truncate_utf8_bytes` routine automatically truncates oversized payloads to this byte boundary. This happens regardless of whether the observation originates from the HTTP API, MCP interface, or internal consolidation hooks.

```rust
// crates/ai-memory-core/src/sanitize.rs
pub const OBSERVATION_BODY_MAX_BYTES: usize = 16 * 1024; // 16,384 bytes

pub fn truncate_utf8_bytes(s: &str, max_bytes: usize) -> &str {
    // Truncation logic ensures valid UTF-8 boundaries
}

```

## HTTP API Character Limits

The web layer adds character-based validation with three tiers of bounds. These are defined in:

- **File:** [`crates/ai-memory-web/src/routes/api.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-web/src/routes/api.rs)

| Constant | Value | Purpose |
|----------|-------|---------|
| `SESSION_OBSERVATIONS_MIN_BODY_CHARS` | **200** | Rejects unreasonably short observations |
| `SESSION_OBSERVATIONS_DEFAULT_BODY_CHARS` | **4,000** | Suggested default for UI implementations |
| `SESSION_OBSERVATIONS_MAX_BODY_CHARS` | **16,384** | Hard ceiling matching the 16 KB byte limit |

When you `POST` to `/workspaces/{ws}/projects/{proj}/sessions/{id}/observations`, the server validates `body.length` against these bounds. Violations return **400 Bad Request** with explicit error messages.

```rust
// crates/ai-memory-web/src/routes/api.rs
const SESSION_OBSERVATIONS_MIN_BODY_CHARS: usize = 200;
const SESSION_OBSERVATIONS_DEFAULT_BODY_CHARS: usize = 4000;
const SESSION_OBSERVATIONS_MAX_BODY_CHARS: usize = 16384;

```

```http
POST /workspaces/default/projects/demo/sessions/12345/observations
Content-Type: application/json

{
  "title": "example",
  "body": "A".repeat(20000),  // Exceeds 16,384 character limit
  "kind": "user-prompt"
}

```

**Response:**

```json
{
  "error": "observation body exceeds maximum allowed size of 16384 characters"
}

```

## Pagination and Result Limits

Fetch operations enforce bounded result sets to prevent memory pressure and network saturation.

From the same [`api.rs`](https://github.com/akitaonrails/ai-memory/blob/main/api.rs) file:

| Constant | Value | Behavior |
|----------|-------|----------|
| `SESSION_OBSERVATIONS_DEFAULT_LIMIT` | **50** | Results returned when no `limit` param specified |
| `SESSION_OBSERVATIONS_MAX_LIMIT` | **200** | Ceiling applied to any `limit` request; values above 200 are clamped |

The `limit` query parameter accepts any integer, but the API internally constrains it to **1–200**. Requesting 500 observations silently returns 200.

```rust
// crates/ai-memory-web/src/routes/api.rs
const SESSION_OBSERVATIONS_DEFAULT_LIMIT: usize = 50;
const SESSION_OBSERVATIONS_MAX_LIMIT: usize = 200;

// Clamping logic in the handler
let limit = query.limit
    .unwrap_or(SESSION_OBSERVATIONS_DEFAULT_LIMIT)
    .clamp(1, SESSION_OBSERVATIONS_MAX_LIMIT);

```

## Enforcement Across Interfaces

The same limits propagate through all entry points:

| Interface | Enforcement Location | Implementation |
|-----------|----------------------|----------------|
| **Core sanitization** | [`crates/ai-memory-core/src/sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs) | `Sanitized::new()` truncates to `OBSERVATION_BODY_MAX_BYTES` |
| **HTTP API** | [`crates/ai-memory-web/src/routes/api.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-web/src/routes/api.rs) | Character validation + pagination clamping |
| **MCP server** | [`crates/ai-memory-mcp/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/server.rs) | Mirrors API constants for `insert_observation` tool |
| **Hooks/consolidation** | [`crates/ai-memory-hooks/src/payload.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/payload.rs) | Applies `OBSERVATION_BODY_MAX_BYTES` when building LLM prompts |

## Practical Code Example

Here's how truncation behaves when inserting an oversized observation through the Rust client:

```rust
use ai_memory_core::{Sanitized, ObservationKind};
use ai_memory_store::Store;

// Construct a 20,000-byte body (exceeds 16 KB limit)
let oversized_body = "x".repeat(20_000);

// Sanitizer automatically truncates to 16,384 bytes
let observation = Sanitized::new(
    ObservationKind::UserPrompt,
    "Truncated Example",
    &oversized_body,
);

// Stored observation contains only first 16 KB
store.insert_observation(observation).await?;

```

The stored record silently contains the truncated content—no error is raised at the core level. The HTTP API, however, rejects rather than truncates oversized submissions.

## Summary

- **16 KB** (`OBSERVATION_BODY_MAX_BYTES`) is the absolute byte ceiling for any observation body, enforced in [`crates/ai-memory-core/src/sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs)
- **16,384 characters** is the matching API ceiling (`SESSION_OBSERVATIONS_MAX_BODY_CHARS`) with a **200-character floor**
- **200 observations** is the maximum retrievable per request (`SESSION_OBSERVATIONS_MAX_LIMIT`), with **50** as the default
- All interfaces—HTTP, MCP, and internal hooks—share these boundaries to maintain datastore integrity

## Frequently Asked Questions

### What happens if I try to store a 100 KB observation through the API?

The server rejects the request with a **400 Bad Request** response stating the body exceeds 16,384 characters. You must truncate or chunk your content client-side before submission.

### Does the 16 KB limit apply to the title or metadata fields?

No. The `OBSERVATION_BODY_MAX_BYTES` constant and related character limits apply strictly to the **`body`** field. Titles and other metadata have separate, less restrictive bounds defined in the validation layer.

### Can I configure these limits at runtime or compile time?

These are **hardcoded constants** throughout the codebase. Changing limits requires modifying the source in [`sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/sanitize.rs) and [`api.rs`](https://github.com/akitaonrails/ai-memory/blob/main/api.rs), then recompiling. There are no environment variable overrides or configuration file hooks exposed.

### Why are there both byte and character limits?

**Byte limits** (`OBSERVATION_BODY_MAX_BYTES`) protect SQLite storage primitives and FTS5 tokenizers from overflow. **Character limits** (`SESSION_OBSERVATIONS_MAX_BODY_CHARS`) provide user-facing validation that aligns with the byte boundary for ASCII/UTF-8 while giving API consumers predictable text-length semantics.