How to Set Up RLM for Long Context Document Processing in Ax

RLM (Recursive Language Model) is Ax's built-in semantic context manager that enables agents to process documents far larger than a single LLM prompt window by splitting work between an iterative JavaScript Actor and a final-response Responder.

When working with the ax-llm/ax framework, you will eventually hit token limits trying to analyze massive documents. RLM solves this by removing long-context fields from the primary prompt, storing them in a persistent JavaScript session, and allowing the Actor to iteratively query sub-agents while the Responder only receives the final synthesized payload.

What Is RLM and Why Use It for Long Documents?

RLM implements a divide-and-conquer pattern through two distinct roles. The Actor runs in a sandboxed JavaScript session that persists across turns, enabling it to read, analyze, and mutate document chunks over multiple iterations. The Responder receives only the final output arguments via final() or ask_clarification() calls, ensuring it never touches the original massive payload.

This architecture matters because it guarantees that neither the Actor's action log nor the Responder's prompt exceeds the LLM's context window, even when processing multi-megabyte documents.

Core Components of the RLM Architecture

Four key pieces work together in src/ax/prompts/rlm.ts and src/ax/funcs/jsRuntime.ts:

• AxRLMConfig – Defines which input fields are treated as long-context (contextFields), which are shared across sub-agents (sharedFields), and how the runtime behaves.
• AxJSRuntime – The sandboxed JavaScript interpreter in src/ax/funcs/jsRuntime.ts that maintains persistent state across Actor turns.
• AxContextManagementConfig – Optional guardrails including errorPruning, hindsightEvaluation, tombstoning, and stateInspection that keep the action log compact during very long tasks.
• agent() factory – Wraps the Actor and Responder definitions into an executable Ax agent.

How RLM Handles Long Context Documents

The system operates through four distinct phases:

1. Input Extraction – Fields listed in contextFields are removed from the LLM prompt and stored in a hidden runtime payload. The Actor accesses this data through the persistent JavaScript session rather than direct prompt injection.

2. Actor Loop – Each turn, the Actor can execute arbitrary JavaScript (variables persist across turns) or call llmQuery() to spawn sub-agents for specific document chunks. The loop terminates when the Actor invokes final() for a successful answer or ask_clarification() for human input.

3. Responder Synthesis – The Responder processes only the payload arguments supplied by the Actor's termination call, generating the final answer without ever seeing the original long-context data.

4. Semantic Context Management – Optional guardrails automatically prune the action log:

   • errorPruning removes failed turns after a successful subsequent turn
   • hindsightEvaluation ranks log entries 0-5 and discards entries below pruneRank
   • tombstoning replaces pruned errors with compact LLM-generated summaries
   • stateInspection injects inspect_runtime() calls when logs exceed contextThreshold characters

Configuring the RLM Agent

To enable long-context processing, configure contextFields and contextManagement when creating your agent:

import {
  AxAIGoogleGeminiModel,
  AxJSRuntime,
  AxJSRuntimePermission,
  agent,
  ai,
} from '@ax-llm/ax';

const llm = ai({
  name: 'google-gemini',
  apiKey: process.env.GOOGLE_APIKEY!,
  config: { model: AxAIGoogleGeminiModel.Gemini3Flash },
});

const analyzer = agent(
  'context:string, query:string -> answer:string, keyFindings:string[] "Analyze a huge document and produce concise findings"',
  {
    contextFields: ['context'],
    runtime: new AxJSRuntime({
      permissions: [AxJSRuntimePermission.TIMING],
    }),
    maxTurns: 30,
    maxLlmCalls: 80,
    mode: 'simple',
    contextManagement: {
      errorPruning: true,
      hindsightEvaluation: true,
      pruneRank: 2,
      tombstoning: true,
      stateInspection: { contextThreshold: 3_000 },
    },
    debug: true,
  }
);

With this configuration, calling analyzer.forward(llm, { context: hugeText, query: '...' }) loads the document into the JavaScript session, allows iterative analysis across up to 30 turns, and automatically prunes the action log to prevent token overflow.

Complete Implementation Examples

Minimal Long-Document Analysis Agent

This example demonstrates the essential setup for processing a massive policy document:

import {
  AxAIOpenAIModel,
  AxJSRuntime,
  AxJSRuntimePermission,
  agent,
  ai,
} from '@ax-llm/ax';

const llm = ai({
  name: 'openai',
  apiKey: process.env.OPENAI_APIKEY!,
  config: { model: AxAIOpenAIModel.Gpt35Turbo },
});

const longDocAgent = agent(
  'doc:string, question:string -> answer:string "Answer a question using the provided huge document"',
  {
    contextFields: ['doc'],
    runtime: new AxJSRuntime({
      permissions: [AxJSRuntimePermission.TIMING],
    }),
    maxTurns: 25,
    maxLlmCalls: 60,
    mode: 'simple',
    contextManagement: {
      errorPruning: true,
      hindsightEvaluation: true,
      pruneRank: 2,
      tombstoning: true,
      stateInspection: { contextThreshold: 4_000 },
    },
  }
);

const hugeText = await fetch('https://example.com/large-policy.txt').then(r => r.text());

const result = await longDocAgent.forward(llm, {
  doc: hugeText,
  question: 'What are the key compliance obligations for data-retention?',
});

console.log('Answer:', result.answer);

The contextFields property ensures the massive text never enters the top-level prompt directly, while the persistent runtime allows you to parse the document once (let rows = parseCSV(doc)) and reuse those variables across multiple turns.

Runtime Inspection for State Management

When processing extremely long documents, the Actor may need to re-ground itself without re-reading the original data. Use inspect_runtime() to capture variable state:

// Inside the Actor code (generated by the LLM)
if (await inspect_runtime().then(snap => snap.length > 3000)) {
  const snapshot = await inspect_runtime();
  console.log('Runtime snapshot:', snapshot);
}

This function returns a compact snapshot of all top-level variables in the JavaScript session. When stateInspection.contextThreshold is configured, Ax automatically injects these calls when the accumulated action log grows too large.

Customizing Tombstone Generation

For production deployments with tight token budgets, customize how pruned error entries are summarized:

contextManagement: {
  tombstoning: {
    model: AxAIOpenAIModel.Gpt35Turbo,
    modelConfig: { temperature: 0, maxTokens: 50 },
  },
},

This configuration forces tombstone generation to use a cheaper, deterministic model and caps summaries at approximately 50 tokens, minimizing overhead while maintaining provenance for the Responder.

Summary

• RLM splits long-context processing between an iterative JavaScript Actor and a lightweight Responder.
• Configure contextFields in AxRLMConfig to remove massive payloads from the primary prompt.
• Enable semantic context management (errorPruning, hindsightEvaluation, tombstoning, stateInspection) to keep action logs within token limits during multi-turn analysis.
• Use AxJSRuntime with minimal permissions to maintain persistent state across Actor turns safely.
• Reference the full implementation in src/examples/rlm-long-task.ts for a working CSV analysis demonstration.

Frequently Asked Questions

What is the difference between contextFields and sharedFields in RLM?

contextFields designates inputs that are removed from the LLM prompt and stored in the JavaScript runtime payload, making them accessible to the Actor without consuming token budget. sharedFields designates inputs that are passed directly to sub-agents spawned via llmQuery() but are not stored in the persistent runtime session.

How does RLM prevent token limit errors during long document processing?

According to the Ax source code in src/ax/prompts/rlm.ts, RLM employs four guardrails: errorPruning removes failed turns after successes, hindsightEvaluation ranks and deletes low-value log entries below a pruneRank threshold, tombstoning compresses removed entries into minimal summaries, and stateInspection triggers inspect_runtime() calls when logs exceed a character threshold, allowing the system to re-ground without accumulating infinite history.

Can I use any LLM provider with RLM, or is it limited to specific models?

RLM is provider-agnostic. The configuration examples use AxAIGoogleGeminiModel and AxAIOpenAIModel interchangeably. As implemented in the agent() factory, RLM only requires a standard Ax LLM interface, meaning you can use OpenAI, Google Gemini, Anthropic Claude, or any other supported provider for both the Actor loop and sub-agent calls.

What permissions should I grant to AxJSRuntime for document processing?

For most long-context document analysis tasks, grant only AxJSRuntimePermission.TIMING, which allows the Actor to measure execution duration without enabling dangerous operations like file system access or network requests. The runtime runs in a sandboxed environment defined in src/ax/funcs/jsRuntime.ts, and restrictive permissions ensure the Actor can manipulate document data in memory safely without side effects.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}