# How Caveman Achieves Token Savings on Agent Outputs and Inputs: 2 Core Mechanisms Explained

> Discover how Caveman slashes token costs with tool schema reduction and context compaction. Learn its 2 core mechanisms for saving on agent inputs and outputs.

- Repository: [Julius Brussee/caveman](https://github.com/JuliusBrussee/caveman)
- Tags: deep-dive
- Published: 2026-09-06

---

**Caveman reduces token costs through tool‑schema reduction and context compaction (summarization), trimming unused tool definitions and replacing long conversation histories with compact summaries.**

The open‑source **Caveman** framework helps developers build LLM agents while actively minimizing token consumption. By implementing two complementary optimizations in its core runtime, Caveman lowers both input and output token bills. This guide examines how these mechanisms work according to the JuliusBrussee/caveman source code, where they live in the codebase, and how to leverage them in your own applications.

## Tool‑Schema Reduction: Sending Only What You Need

Most agent frameworks transmit the complete catalog of available tools on every turn—even when only a subset is relevant. Caveman takes a smarter approach.

### How Tool‑Schema Reduction Works

In [`packages/sdk/typescript/src/index.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/sdk/typescript/src/index.ts), the SDK computes a **reduced set of tools** actually required for the current request and forwards only those definitions to the model. The full and reduced catalogs are tracked separately:

- `tokensBefore`: token count of the full tool catalog
- `tokensAfter`: token count of the reduced catalog
- `tokensSaved`: the computed delta between them

This logic appears in lines 22–31 of the TypeScript SDK index, where the `RunResult` interface exposes these fields to callers.

### Why This Saves Money

Trimming unused tool definitions directly shrinks the payload sent to the LLM. Since providers bill by token count, every eliminated definition lowers the input cost. The client receives concrete visibility into these savings through the SDK's public API.

```ts
interface RunResult {
  /** Tokens before the reduction (full catalog). */
  tokensBefore: number;
  /** Tokens after the reduction (reduced catalog). */
  tokensAfter: number;
  /** Tokens actually saved by the reduction. */
  tokensSaved: number;
  /** Inferred tokens saved by a compaction summary. */
  tokensSavedInferred?: number;
}

```

## Context Compaction: Summarizing to Stay Within Budget

When conversations grow long, agents risk hitting context window limits. Caveman solves this through **context compaction**—an intelligent summarization process that replaces raw message history with a compact representation.

### The Compaction Engine Architecture

The compaction system lives in [`packages/agent/src/compaction.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/agent/src/compaction.ts) (lines 6–25 and 45–51). It implements a **budget‑aware algorithm** that:

1. Monitors session token count against the model's context limit
2. Evicts old messages when thresholds are exceeded
3. Invokes a summarizer LLM to produce a concise state representation
4. Reserves tokens for both the summarize call and subsequent working call

### Token Savings Calculation

The engine models cost explicitly through `CompactionOptions`:

- `tokensBefore`: tokens in the original history
- `tokensAfter`: tokens in the compacted summary
- `tokensSavedInferred`: estimated net reduction

Compaction only triggers when projected savings outweigh the cost of the extra summarize request. This guardrail prevents wasteful summarization of short contexts.

### Why Summarization Beats Truncation

Raw conversation history often spans thousands of tokens. A well‑crafted summary captures essential state in a few hundred tokens. The net effect is a **negative token delta** for the turn—reported as `tokensSavedInferred` in run results. This mechanism appears in the code aggregation logic at [`packages/agent/src/code.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/agent/src/code.ts) (lines 604–821).

## Practical Implementation: Using Token Savings Features

### Inspecting Savings via the TypeScript SDK

```ts
import { Cave } from "@caveman/sdk";

async function demo() {
  const cave = new Cave({ 
    baseURL: "https://cave.example.com", 
    apiKey: "CAVE_API_KEY" 
  });

  const result = await cave.run({
    model: "gpt-5.5",
    messages: [{ role: "user", content: "Explain quantum entanglement." }],
  });

  console.log(`Input tokens: ${result.tokensBefore}`);
  console.log(`Reduced tokens: ${result.tokensAfter}`);
  console.log(`Tokens saved by schema reduction: ${result.tokensSaved}`);

  if (result.tokensSavedInferred) {
    console.log(`Additional tokens saved by compaction: ${result.tokensSavedInferred}`);
  }
}

```

The `result` object surfaces both **schema savings** (`tokensSaved`) and **compaction savings** (`tokensSavedInferred`).

### Configuring Compaction Behavior

```ts
import { Cave, CompactionOptions } from "@caveman/sdk";

const compactionOpts: CompactionOptions = {
  minTokensToCompact: 2048,  // Trigger threshold
  reserveTokens: 300,        // Budget for summarizer call
};

await cave.run({
  model: "gpt-5.5",
  messages: longHistory,
  compaction: compactionOpts,
});

```

Adjusting `CompactionOptions` lets you tune when summarization occurs and how aggressively tokens are reclaimed.

### Monitoring via CLI

```bash
caveman run --model gpt-5.5 --prompt "Explain recursion" \
  | grep "estimated tokens saved"

```

Sample output:

```

· 452 estimated tokens saved (inferred; counter basis estimated_bytes_div_4)

```

This formatting logic resides in [`packages/cli/src/index.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/cli/src/index.ts) (lines 10044–10045).

## Budget Enforcement: Preventing Overruns While Saving

The [`packages/agent/src/budget.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/agent/src/budget.ts) module implements **token‑budget enforcement** with built‑in reservation for compaction calls. This ensures agents:

- Never exceed allocated token quotas
- Still trigger compaction when savings are favorable
- Maintain predictable cost profiles

The budget system cooperates with the compaction engine to reserve tokens upfront, eliminating surprise overages from summarize operations.

## Summary

- **Tool‑schema reduction** in [`packages/sdk/typescript/src/index.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/sdk/typescript/src/index.ts) computes minimal tool sets, cutting input tokens before they reach the model
- **Context compaction** in [`packages/agent/src/compaction.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/agent/src/compaction.ts) summarizes long histories, replacing thousands of tokens with compact state representations
- **Budget reservation** in [`packages/agent/src/budget.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/agent/src/budget.ts) safeguards against overruns while permitting aggressive optimization
- Both mechanisms expose metrics through `RunResult` (`tokensSaved`, `tokensSavedInferred`) for cost tracking and optimization

## Frequently Asked Questions

### How do I see exactly how many tokens Caveman saved on a request?

The SDK's `RunResult` object contains `tokensSaved` for schema reduction and `tokensSavedInferred` for compaction gains. Both fields populate automatically on every `cave.run()` call.

### Does compaction always reduce token costs?

No. The compaction engine in [`packages/agent/src/compaction.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/agent/src/compaction.ts) only triggers when `tokensSavedInferred` is projected to be positive—that is, when summary cost is lower than the history it replaces. Short conversations skip compaction entirely.

### Can I disable token‑saving features for debugging?

While the SDK doesn't expose a global "disable optimization" flag, you can effectively bypass compaction by setting `minTokensToCompact` to a very high value in `CompactionOptions`. Schema reduction is integral to the SDK's request pipeline.

### What happens if the summarizer itself exceeds the reserved budget?

The budget system in [`packages/agent/src/budget.ts`](https://github.com/JuliusBrussee/caveman/blob/main/packages/agent/src/budget.ts) reserves tokens specifically for the compaction call. If a summarizer overruns, the error propagates to the caller—preventing silent cost explosions while maintaining transparency.