# How to Set Up RLM for Long Context Document Processing in Ax

> Master RLM setup for extensive document processing in Ax LLM. Leverage its semantic context manager and iterative actor-responder architecture for superior long-context handling. Get started today.

- Repository: [Ax/ax](https://github.com/ax-llm/ax)
- Tags: how-to-guide
- Published: 2026-02-25

---

**RLM (Recursive Language Model)** is Ax's built-in semantic context manager that enables agents to process documents far larger than a single LLM prompt window by splitting work between an iterative JavaScript Actor and a final-response Responder.

When working with the [ax-llm/ax](https://github.com/ax-llm/ax) framework, you will eventually hit token limits trying to analyze massive documents. RLM solves this by removing long-context fields from the primary prompt, storing them in a persistent JavaScript session, and allowing the Actor to iteratively query sub-agents while the Responder only receives the final synthesized payload.

## What Is RLM and Why Use It for Long Documents?

RLM implements a divide-and-conquer pattern through two distinct roles. The **Actor** runs in a sandboxed JavaScript session that persists across turns, enabling it to read, analyze, and mutate document chunks over multiple iterations. The **Responder** receives only the final output arguments via `final()` or `ask_clarification()` calls, ensuring it never touches the original massive payload.

This architecture matters because it guarantees that neither the Actor's action log nor the Responder's prompt exceeds the LLM's context window, even when processing multi-megabyte documents.

## Core Components of the RLM Architecture

Four key pieces work together in [`src/ax/prompts/rlm.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/prompts/rlm.ts) and [`src/ax/funcs/jsRuntime.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/funcs/jsRuntime.ts):

- **`AxRLMConfig`** – Defines which input fields are treated as long-context (`contextFields`), which are shared across sub-agents (`sharedFields`), and how the runtime behaves.
- **`AxJSRuntime`** – The sandboxed JavaScript interpreter in [`src/ax/funcs/jsRuntime.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/funcs/jsRuntime.ts) that maintains persistent state across Actor turns.
- **`AxContextManagementConfig`** – Optional guardrails including `errorPruning`, `hindsightEvaluation`, `tombstoning`, and `stateInspection` that keep the action log compact during very long tasks.
- **`agent()` factory** – Wraps the Actor and Responder definitions into an executable Ax agent.

## How RLM Handles Long Context Documents

The system operates through four distinct phases:

1. **Input Extraction** – Fields listed in `contextFields` are removed from the LLM prompt and stored in a hidden runtime payload. The Actor accesses this data through the persistent JavaScript session rather than direct prompt injection.

2. **Actor Loop** – Each turn, the Actor can execute arbitrary JavaScript (variables persist across turns) or call `llmQuery()` to spawn sub-agents for specific document chunks. The loop terminates when the Actor invokes `final()` for a successful answer or `ask_clarification()` for human input.

3. **Responder Synthesis** – The Responder processes only the payload arguments supplied by the Actor's termination call, generating the final answer without ever seeing the original long-context data.

4. **Semantic Context Management** – Optional guardrails automatically prune the action log:
   - **errorPruning** removes failed turns after a successful subsequent turn
   - **hindsightEvaluation** ranks log entries 0-5 and discards entries below `pruneRank`
   - **tombstoning** replaces pruned errors with compact LLM-generated summaries
   - **stateInspection** injects `inspect_runtime()` calls when logs exceed `contextThreshold` characters

## Configuring the RLM Agent

To enable long-context processing, configure `contextFields` and `contextManagement` when creating your agent:

```typescript
import {
  AxAIGoogleGeminiModel,
  AxJSRuntime,
  AxJSRuntimePermission,
  agent,
  ai,
} from '@ax-llm/ax';

const llm = ai({
  name: 'google-gemini',
  apiKey: process.env.GOOGLE_APIKEY!,
  config: { model: AxAIGoogleGeminiModel.Gemini3Flash },
});

const analyzer = agent(
  'context:string, query:string -> answer:string, keyFindings:string[] "Analyze a huge document and produce concise findings"',
  {
    contextFields: ['context'],
    runtime: new AxJSRuntime({
      permissions: [AxJSRuntimePermission.TIMING],
    }),
    maxTurns: 30,
    maxLlmCalls: 80,
    mode: 'simple',
    contextManagement: {
      errorPruning: true,
      hindsightEvaluation: true,
      pruneRank: 2,
      tombstoning: true,
      stateInspection: { contextThreshold: 3_000 },
    },
    debug: true,
  }
);

```

With this configuration, calling `analyzer.forward(llm, { context: hugeText, query: '...' })` loads the document into the JavaScript session, allows iterative analysis across up to 30 turns, and automatically prunes the action log to prevent token overflow.

## Complete Implementation Examples

### Minimal Long-Document Analysis Agent

This example demonstrates the essential setup for processing a massive policy document:

```typescript
import {
  AxAIOpenAIModel,
  AxJSRuntime,
  AxJSRuntimePermission,
  agent,
  ai,
} from '@ax-llm/ax';

const llm = ai({
  name: 'openai',
  apiKey: process.env.OPENAI_APIKEY!,
  config: { model: AxAIOpenAIModel.Gpt35Turbo },
});

const longDocAgent = agent(
  'doc:string, question:string -> answer:string "Answer a question using the provided huge document"',
  {
    contextFields: ['doc'],
    runtime: new AxJSRuntime({
      permissions: [AxJSRuntimePermission.TIMING],
    }),
    maxTurns: 25,
    maxLlmCalls: 60,
    mode: 'simple',
    contextManagement: {
      errorPruning: true,
      hindsightEvaluation: true,
      pruneRank: 2,
      tombstoning: true,
      stateInspection: { contextThreshold: 4_000 },
    },
  }
);

const hugeText = await fetch('https://example.com/large-policy.txt').then(r => r.text());

const result = await longDocAgent.forward(llm, {
  doc: hugeText,
  question: 'What are the key compliance obligations for data-retention?',
});

console.log('Answer:', result.answer);

```

The `contextFields` property ensures the massive text never enters the top-level prompt directly, while the persistent runtime allows you to parse the document once (`let rows = parseCSV(doc)`) and reuse those variables across multiple turns.

### Runtime Inspection for State Management

When processing extremely long documents, the Actor may need to re-ground itself without re-reading the original data. Use `inspect_runtime()` to capture variable state:

```typescript
// Inside the Actor code (generated by the LLM)
if (await inspect_runtime().then(snap => snap.length > 3000)) {
  const snapshot = await inspect_runtime();
  console.log('Runtime snapshot:', snapshot);
}

```

This function returns a compact snapshot of all top-level variables in the JavaScript session. When `stateInspection.contextThreshold` is configured, Ax automatically injects these calls when the accumulated action log grows too large.

### Customizing Tombstone Generation

For production deployments with tight token budgets, customize how pruned error entries are summarized:

```typescript
contextManagement: {
  tombstoning: {
    model: AxAIOpenAIModel.Gpt35Turbo,
    modelConfig: { temperature: 0, maxTokens: 50 },
  },
},

```

This configuration forces tombstone generation to use a cheaper, deterministic model and caps summaries at approximately 50 tokens, minimizing overhead while maintaining provenance for the Responder.

## Summary

- **RLM** splits long-context processing between an iterative JavaScript Actor and a lightweight Responder.
- Configure `contextFields` in `AxRLMConfig` to remove massive payloads from the primary prompt.
- Enable **semantic context management** (`errorPruning`, `hindsightEvaluation`, `tombstoning`, `stateInspection`) to keep action logs within token limits during multi-turn analysis.
- Use `AxJSRuntime` with minimal permissions to maintain persistent state across Actor turns safely.
- Reference the full implementation in [`src/examples/rlm-long-task.ts`](https://github.com/ax-llm/ax/blob/main/src/examples/rlm-long-task.ts) for a working CSV analysis demonstration.

## Frequently Asked Questions

### What is the difference between `contextFields` and `sharedFields` in RLM?

`contextFields` designates inputs that are removed from the LLM prompt and stored in the JavaScript runtime payload, making them accessible to the Actor without consuming token budget. `sharedFields` designates inputs that are passed directly to sub-agents spawned via `llmQuery()` but are not stored in the persistent runtime session.

### How does RLM prevent token limit errors during long document processing?

According to the Ax source code in [`src/ax/prompts/rlm.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/prompts/rlm.ts), RLM employs four guardrails: **errorPruning** removes failed turns after successes, **hindsightEvaluation** ranks and deletes low-value log entries below a `pruneRank` threshold, **tombstoning** compresses removed entries into minimal summaries, and **stateInspection** triggers `inspect_runtime()` calls when logs exceed a character threshold, allowing the system to re-ground without accumulating infinite history.

### Can I use any LLM provider with RLM, or is it limited to specific models?

RLM is provider-agnostic. The configuration examples use `AxAIGoogleGeminiModel` and `AxAIOpenAIModel` interchangeably. As implemented in the `agent()` factory, RLM only requires a standard Ax LLM interface, meaning you can use OpenAI, Google Gemini, Anthropic Claude, or any other supported provider for both the Actor loop and sub-agent calls.

### What permissions should I grant to `AxJSRuntime` for document processing?

For most long-context document analysis tasks, grant only `AxJSRuntimePermission.TIMING`, which allows the Actor to measure execution duration without enabling dangerous operations like file system access or network requests. The runtime runs in a sandboxed environment defined in [`src/ax/funcs/jsRuntime.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/funcs/jsRuntime.ts), and restrictive permissions ensure the Actor can manipulate document data in memory safely without side effects.