# Implementing Context Engineering Tools for Enhanced Claude Interactions: A Technical Guide

> Master context engineering tools for Claude agents. Learn to process over 300K tokens within the 1M token window using Anthropic API primitives for efficient long-running interactions.

- Repository: [Anthropic/claude-cookbooks](https://github.com/anthropics/claude-cookbooks)
- Tags: how-to-guide
- Published: 2026-04-14

---

**Anthropic provides three API primitives—compaction, tool-result clearing, and structured memory—that enable developers to maintain long-running Claude agents processing over 300,000 tokens without exhausting the 1 million token context window.**

The `anthropics/claude-cookbooks` repository demonstrates production-ready patterns for managing Claude’s context window in complex agent workflows. This guide analyzes the source code behind a synthetic biology research agent that ingests a 320,000-token corpus, showing you how to implement **conversation compaction**, **tool-result clearing**, and **persistent memory tools** using the actual API configurations and Python handlers from the repository.

## Core Context Management Primitives

The Anthropic API exposes three beta features specifically designed for context engineering. According to the implementation in `tool_use/context_engineering/context_engineering_tools.ipynb`, these primitives can be combined to create agents that never hit the token ceiling while retaining critical reasoning.

### Conversation Compaction (`compact_20260112`)

**Compaction** summarizes the entire conversation history into a condensed representation when the context approaches a specified threshold. When triggered—typically at 150,000 tokens—the API replaces the full message history with a summary that preserves essential findings while discarding low-signal tokens.

The configuration accepts a `trigger` token count and custom `instructions` for what to preserve:

```python
compaction_cfg = {
    "compact_20260112": {
        "trigger": 150_000,
        "instructions": "Summarise all findings; keep organism names and numeric results.",
    }
}

```

As implemented in the research agent example, this primitive reduces the context window from approximately 335,000 tokens down to roughly 70,000 tokens while maintaining the scientific conclusions necessary for subsequent analysis.

### Tool-Result Clearing (`clear_tool_uses_20250919`)

**Tool-result clearing** replaces large `tool_result` content—such as file reads or API responses—with tiny placeholders while preserving the `tool_use` block to maintain provenance. This is distinct from compaction because it operates on individual tool results rather than the entire conversation.

The primitive supports selective retention via the `keep` parameter (number of recent results to preserve) and `exclude_tools` (tool names that should never be cleared):

```python
clearing_cfg = {
    "clear_tool_uses_20250919": {
        "trigger": 100_000,
        "keep": 3,
        "exclude_tools": ["memory"],
    }
}

```

In the `claude-cookbooks` demonstration, enabling clearing alone keeps the research agent’s context under 180,000 tokens despite reading eight large review documents totaling 320,000 tokens.

### Persistent Memory Tool (`memory_20250818`)

The **memory** primitive moves structured data outside the context window entirely, enabling multi-session persistence. Unlike the automatic compaction and clearing features, memory is invoked explicitly as a tool call that CRUDs files in a sandboxed directory.

The `MemoryToolHandler` class in [`tool_use/memory_tool.py`](https://github.com/anthropics/claude-cookbooks/blob/main/tool_use/memory_tool.py) implements six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) with path traversal protection:

```python
def _validate_path(self, path: str) -> Path:
    if not path.startswith("/memories"):
        raise ValueError("Path must start with /memories")
    relative_path = path[len("/memories"):].lstrip("/")
    full_path = (self.memory_root / relative_path).resolve()
    full_path.relative_to(self.memory_root.resolve())
    return full_path

```

This validation ensures agents can only access files within the designated memory root, preventing directory traversal attacks while allowing arbitrary note-taking across sessions.

## Agent Architecture Deep Dive

The research agent architecture separates concerns between data provision, execution orchestration, and state management. Understanding these components is essential for implementing context engineering in production applications.

### Synthetic Corpus ([`research_corpus.py`](https://github.com/anthropics/claude-cookbooks/blob/main/research_corpus.py))

The agent operates against a deterministic, in-memory corpus defined in [`tool_use/context_engineering/research_corpus.py`](https://github.com/anthropics/claude-cookbooks/blob/main/tool_use/context_engineering/research_corpus.py). This dictionary maps virtual file paths to content strings totaling approximately 320,000 tokens:

```python
CORPUS: dict[str, str] = {
    "/research/celegans_review.md": """# Model Organism Review: C. elegans ...""",

    "/research/drosophila_review.md": """# Model Organism Review: Drosophila ...""",

    # … 6 additional organisms …

}

```

Using synthetic data eliminates external I/O latency and guarantees reproducible token counts for testing context management strategies.

### Execution Loop (`context_engineering_tools.ipynb`)

The `run_research_session` function orchestrates the agent loop, handling API calls with optional context management configurations:

```python
def run_research_session(
    initial_prompt: str,
    *,
    context_management: dict | None = None,
    betas: list[str] | None = None,
    memory_handler=None,
    max_turns: int = 12,
) -> SessionResult:
    tools = list(BASE_TOOLS)
    if memory_handler is not None:
        tools.append(MEMORY_TOOL_SPEC)
    
    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=4096,
        system=SYSTEM_PROMPT,
        messages=messages,
        tools=tools,
        context_management=context_management,
        betas=betas,
    )

```

The function tracks `token_trajectory` and `events` to visualize when compaction or clearing triggers fire during the session.

## Practical Implementation Examples

The following patterns demonstrate how to configure the Anthropic API to handle increasingly complex context management scenarios.

### Baseline Without Context Management

Running the agent without optimization quickly exhausts the available context:

```python
from context_engineering_tools import run_research_session

baseline = run_research_session(
    RESEARCH_TASK,
    label="baseline",
    max_turns=12,
)

```

*Result:* The agent reaches approximately 335,000 tokens in five turns, storing every file read and reasoning step in the context window.

### Implementing Tool-Result Clearing

To reduce memory pressure, configure the `clear_tool_uses_20250919` beta identifier:

```python
clearing_cfg = {
    "clear_tool_uses_20250919": {
        "trigger": 100_000,
        "keep": 3,
        "exclude_tools": ["memory"],
    }
}

clearing = run_research_session(
    RESEARCH_TASK,
    label="clearing",
    context_management=clearing_cfg,
)

```

*Result:* Earlier `read_file` results are replaced with placeholders, keeping the context under 180,000 tokens while preserving the record that files were accessed.

### Combining Clearing and Compaction

For maximum efficiency, merge both configurations to handle both large tool outputs and accumulated reasoning:

```python
compaction_cfg = {
    "compact_20260112": {
        "trigger": 150_000,
        "instructions": "Summarise all findings; keep organism names and numeric results.",
    }
}
combined_cfg = {**clearing_cfg, **compaction_cfg}

combined = run_research_session(
    RESEARCH_TASK,
    label="combined",
    context_management=combined_cfg,
)

```

*Result:* After turn three, the context compacts to approximately 70,000 tokens. Subsequent tool calls stay below 200,000 tokens despite continuing the full research synthesis.

### Persisting State Across Sessions

Use the `MemoryToolHandler` to maintain notes between distinct API sessions:

```python
from tool_use.memory_tool import MemoryToolHandler

mem_handler = MemoryToolHandler()  # Writes to ./memory_storage/memories

# Session 1: Record findings

session1 = run_research_session(RESEARCH_TASK, memory_handler=mem_handler)
mem_handler.execute(
    command="create",
    path="/memories/batch1_notes.txt",
    file_text="\n".join(session1.notes),
)

# Session 2: Retrieve previous context

session2 = run_research_session(
    "Continue analysis using previous notes",
    memory_handler=mem_handler,
)

```

The memory tool writes to the filesystem, allowing arbitrarily large notes to persist indefinitely without consuming API tokens.

## Summary

- **Conversation compaction** (`compact_20260112`) triggers at a token threshold to summarize the entire message history, reducing context size by 50-80% while preserving reasoning.
- **Tool-result clearing** (`clear_tool_uses_20250919`) replaces large tool outputs with placeholders, configured via `keep` and `exclude_tools` parameters to protect recent or critical results.
- **Memory tools** (`memory_20250818`) implement external state persistence through the `MemoryToolHandler` class, supporting CRUD operations validated against path traversal.
- The `claude-cookbooks` implementation demonstrates processing 320,000 tokens of synthetic research data while maintaining context under 75,000 tokens using combined primitives.

## Frequently Asked Questions

### What is the difference between compaction and tool-result clearing in the Anthropic API?

**Compaction** operates on the entire conversation history, generating a summary that replaces all previous messages when the token count exceeds the trigger threshold. **Tool-result clearing** specifically targets individual `tool_result` blocks, replacing the content of old tool calls with placeholders while keeping the `tool_use` record intact. Use compaction when the conversation itself grows too large; use clearing when tool outputs (like file reads or API responses) dominate the token count but the conversation history remains manageable.

### How do I prevent the memory tool from accessing files outside its designated directory?

The `MemoryToolHandler` in [`tool_use/memory_tool.py`](https://github.com/anthropics/claude-cookbooks/blob/main/tool_use/memory_tool.py) implements path validation via the `_validate_path` method, which checks that all paths start with `/memories` and uses `pathlib.Path.resolve()` with `relative_to()` to ensure the resolved path remains within the `memory_root`. Any attempt to traverse to parent directories raises a `ValueError` before file operations execute.

### Can I use context management features with any Claude model?

Context engineering primitives like `compact_20260112` and `clear_tool_uses_20250919` are beta features that must be explicitly enabled via the `betas` or `context_management` parameters in the API request. According to the `claude-cookbooks` source, these features work with Claude Sonnet and Claude Opus models where the context window limit is a concern, but you should verify current availability in the Anthropic documentation as beta identifiers are versioned with date suffixes (e.g., `_20260112`).

### What happens if a compaction trigger fires during a complex multi-step reasoning task?

When the `compact_20260112` trigger fires, the API immediately summarizes the conversation up to that point according to your `instructions` parameter. As demonstrated in the research agent notebook, providing specific instructions like "keep organism names and numeric results" ensures that critical in-progress reasoning is preserved in the summary. The agent continues from the compacted state, maintaining awareness of previous conclusions despite the truncated history.