# Goose Context Revision Algorithm: Token Management and Cost Optimization Explained

> Explore Goose's context revision algorithm. Optimize token management, reduce API costs with LLM summarization, and manage context window limits effectively. Discover efficient large language model interaction.

- Repository: [Block Open Source/goose](https://github.com/block/goose)
- Tags: deep-dive
- Published: 2026-04-05

---

**Goose employs a layered context revision algorithm that monitors token usage ratios against configurable thresholds, automatically compacts conversation history through LLM summarization, and progressively prunes tool responses to maintain hard context window limits while minimizing API costs.**

The `block/goose` repository implements this sophisticated context revision algorithm to prevent LLM context window overflow while optimizing token costs. Written in Rust, the system proactively manages conversation history through intelligent compaction and background summarization of tool interactions, ensuring the prompt stays within the model's `context_limit` at the lowest possible token cost.

## Token Tracking and Session Management

Goose maintains exact token counts per session through the `Session` struct defined in [`crates/goose/src/session/session_manager.rs`](https://github.com/block/goose/blob/main/crates/goose/src/session/session_manager.rs). This struct tracks both current turn usage and accumulated totals across the entire conversation lifecycle.

```rust
pub struct Session {
    pub total_tokens: Option<i32>,
    pub input_tokens: Option<i32>,
    pub output_tokens: Option<i32>,
    pub accumulated_total_tokens: Option<i32>,
    pub accumulated_input_tokens: Option<i32>,
    pub accumulated_output_tokens: Option<i32>,
}

```

When a provider finishes a request, `ProviderUsage` is merged into the `Session`, ensuring Goose always knows the exact token cost of each turn. This precise bookkeeping enables the context revision algorithm to make data-driven decisions about when to trigger compaction.

## Detecting Context Window Limits

The algorithm detects imminent overflow through the `check_if_compaction_needed` function in [`crates/goose/src/context_mgmt/mod.rs`](https://github.com/block/goose/blob/main/crates/goose/src/context_mgmt/mod.rs). This function compares the current token ratio against the `GOOSE_AUTO_COMPACT_THRESHOLD` configuration parameter, which defaults to **0.8** (80%).

```rust
pub async fn check_if_compaction_needed(
    provider: &dyn Provider,
    conversation: &Conversation,
    threshold_override: Option<f64>,
    session: &crate::session::Session,
) -> Result<bool> {
    let threshold = threshold_override.unwrap_or_else(|| {
        Config::global()
            .get_param::<f64>("GOOSE_AUTO_COMPACT_THRESHOLD")
            .unwrap_or(DEFAULT_COMPACTION_THRESHOLD)
    });

    let context_limit = provider.get_model_config().context_limit();

    let (current_tokens, _) = match session.total_tokens {
        Some(tokens) => (tokens as usize, "session metadata"),
        None => {
            let token_counter = create_token_counter().await?;
            let token_counts: Vec<_> = conversation.messages()
                .iter()
                .filter(|m| m.is_agent_visible())
                .map(|msg| token_counter.count_chat_tokens("", std::slice::from_ref(msg), &[]))
                .collect();
            (token_counts.iter().sum(), "estimated")
        }
    };

    let usage_ratio = current_tokens as f64 / context_limit as f64;
    Ok(usage_ratio > threshold && threshold > 0.0 && threshold < 1.0)
}

```

If the `usage_ratio` exceeds the threshold, Goose immediately initiates the compaction phase to prevent context window exhaustion.

## Computing Safe Tool-Call Budgets

Before compaction, the algorithm calculates how many tool-call messages to preserve using `compute_tool_call_cutoff` in [`crates/goose/src/context_mgmt/mod.rs`](https://github.com/block/goose/blob/main/crates/goose/src/context_mgmt/mod.rs). This translates the model's `context_limit` and compaction threshold into a concrete message count.

```rust
pub fn compute_tool_call_cutoff(context_limit: usize, compaction_threshold: f64) -> usize {
    let threshold = if compaction_threshold > 0.0 && compaction_threshold <= 1.0 {
        compaction_threshold
    } else {
        DEFAULT_COMPACTION_THRESHOLD
    };
    let effective_limit = (context_limit as f64 * threshold) as usize;
    (3 * effective_limit / 20_000).clamp(10, 500)
}

```

The resulting value represents the maximum number of old tool-call messages Goose retains before summarizing them into compact descriptions. The formula scales the effective token budget down to a manageable message count, clamped between 10 and 500 messages.

## Conversation Compaction Strategies

When threshold detection triggers compaction, Goose employs a two-tiered fallback strategy to ensure the conversation fits within the provider's limits.

### Full History Summarization

The `compact_messages` function handles the primary compaction workflow. It preserves the most recent user message (unless manually compacting), then calls `do_compact` to generate a summary of the visible history.

```rust
pub async fn compact_messages(
    provider: &dyn Provider,
    session_id: &str,
    conversation: &Conversation,
    manual_compact: bool,
) -> Result<(Conversation, ProviderUsage)> {
    // 1️⃣ Preserve the most recent user message (unless manual)
    // 2️⃣ Call `do_compact` → provider creates a short summary of the whole visible history
    // 3️⃣ Re‑assemble the conversation:
    //    - All old messages become *agent‑invisible* (kept for accounting but not sent)
    //    - The new summary becomes *agent‑only*
    //    - A continuation message tells the model to keep talking naturally
    // 4️⃣ If a user message was preserved, re‑append its text as a fresh user turn
}

```

The result is a new `Conversation` where old messages become **agent-invisible** (retained for accounting but excluded from the LLM prompt), replaced by a concise summary and continuation message that maintains conversational coherence.

### Progressive Removal Fallback

If the provider still reports a `ContextLengthExceeded` error after initial compaction, `do_compact` implements progressive pruning. The algorithm iteratively strips increasing percentages of tool-response messages from the middle of the visible set.

```rust
let removal_percentages = [0, 10, 20, 50, 100];
for (attempt, &remove_percent) in removal_percentages.iter().enumerate() {
    let filtered_messages = filter_tool_responses(&agent_visible_messages, remove_percent);
    let system_prompt = render_template("compaction.md", &SummarizeContext { messages })?;

    match provider.complete_fast(session_id, &system_prompt, &summarization_request, &[]) {
        Ok((mut response, mut usage)) => { /* success */ }
        Err(e) if matches!(e, ProviderError::ContextLengthExceeded(_)) => {
            continue;
        }
        Err(e) => return Err(e.into()),
    }
}

```

This **progressive removal** attempts 0%, 10%, 20%, 50%, and finally 100% removal of tool responses until the provider accepts the request, guaranteeing hard safety against context overflow.

## Background Tool-Call Summarization

Beyond full compaction, Goose optimizes token costs by summarizing individual tool-call pairs in the background. When the number of stored tool calls exceeds the computed cutoff, `maybe_summarize_tool_pairs` spawns async tasks to compress old pairs.

```rust
pub fn maybe_summarize_tool_pairs(
    provider: Arc<dyn Provider>,
    session_id: String,
    conversation: Conversation,
    cutoff: usize,
    protect_last_n: usize,
) -> JoinHandle<Vec<(Message, String)>> {
    // If tool‑pair summarisation is enabled and the provider does NOT manage its own context,
    // compute which tool‑ids are eligible and spawn async tasks calling `summarize_tool_call`
}

```

The `summarize_tool_call` function gathers all messages belonging to a specific tool call and its response, then prompts the provider to generate a one-sentence description.

```rust
pub async fn summarize_tool_call(
    provider: &dyn Provider,
    session_id: &str,
    conversation: &Conversation,
    tool_id: &str,
) -> Result<Message> {
    // Gather all messages that belong to the tool call & its response
    // Build a concise user message containing the formatted history
    // Ask the provider to produce a short description
}

```

These summaries replace bulky JSON payloads with minimal text, significantly reducing token counts for tool-heavy conversations while preserving essential semantic information.

## Complete Workflow Example

The following integration demonstrates the complete context revision algorithm in action, combining threshold detection, compaction, and background summarization:

```rust
// 1️⃣ Check if compaction is needed based on current token ratio
let needs_compact = check_if_compaction_needed(
    provider.as_ref(),
    &conversation,
    None,
    &session,
).await?;

// 2️⃣ Compact the entire history if threshold exceeded
if needs_compact {
    let (compact_conv, usage) = compact_messages(
        provider.as_ref(),
        &session_id,
        &conversation,
        false, // automatic compaction
    ).await?;
    conversation = compact_conv;
    session = session.with_input_tokens(Some(usage.input_tokens()));
}

// 3️⃣ Compute tool-call cutoff for background summarization
let cutoff = compute_tool_call_cutoff(
    provider.get_model_config().context_limit(),
    Config::global()
        .get_param::<f64>("GOOSE_AUTO_COMPACT_THRESHOLD")
        .unwrap_or(DEFAULT_COMPACTION_THRESHOLD),
);

// 4️⃣ Spawn background task to summarize old tool pairs
let summariser = maybe_summarize_tool_pairs(
    provider.clone(),
    session_id.clone(),
    conversation.clone(),
    cutoff,
    2, // protect last 2 tool calls
);

// 5️⃣ Integrate summaries back into conversation
let tool_summaries = summariser.await?;
for (summary_msg, tool_id) in tool_summaries {
    conversation = conversation.with_inserted_summary(summary_msg);
}

```

This workflow ensures **hard safety** through `compact_messages`, **cost efficiency** by minimizing visible tokens, and **semantic preservation** by protecting recent user turns and necessary tool results.

## Summary

Goose's context revision algorithm combines multiple strategies to maintain optimal token usage:

- **Exact Token Tracking**: The `Session` struct in [`session_manager.rs`](https://github.com/block/goose/blob/main/session_manager.rs) maintains precise input/output token counts merged from `ProviderUsage` after each request.
- **Threshold-Based Triggers**: The `check_if_compaction_needed` function monitors the ratio of current tokens to `context_limit`, triggering compaction when exceeding the default 80% threshold.
- **Safe Budget Calculation**: `compute_tool_call_cutoff` determines the maximum number of tool-call messages to retain before summarization, scaling the context window down to a clamped range of 10-500 messages.
- **Progressive Fallback**: The `do_compact` function implements aggressive pruning (10% → 20% → 50% → 100%) if providers still report context length errors.
- **Background Optimization**: `maybe_summarize_tool_pairs` spawns async tasks to replace old tool-call pairs with concise descriptions, running parallel to the main LLM request.

## Frequently Asked Questions

### What triggers the context revision algorithm in Goose?

The algorithm triggers when the `check_if_compaction_needed` function detects that the current token count divided by the model's `context_limit` exceeds the `GOOSE_AUTO_COMPACT_THRESHOLD`, which defaults to **0.8** (80%). This calculation uses precise token counts from the `Session` struct or falls back to estimation via `count_chat_tokens` in [`crates/goose/src/token_counter.rs`](https://github.com/block/goose/blob/main/crates/goose/src/token_counter.rs) when metadata is unavailable.

### How does Goose handle persistent context length errors from providers?

If the provider returns a `ContextLengthExceeded` error during compaction, Goose enters a progressive removal loop inside `do_compact`. The algorithm iteratively filters out 10%, then 20%, then 50%, and finally 100% of tool-response messages from the middle of the visible conversation, attempting summarization at each stage until the request succeeds or all tool responses are removed.

### What is the default auto-compact threshold and how is it configured?

The default threshold is **0.8**, meaning compaction initiates when token usage exceeds 80% of the model's context window. Users can override this via the `GOOSE_AUTO_COMPACT_THRESHOLD` configuration parameter, which `check_if_compaction_needed` retrieves through `Config::global().get_param::<f64>()`.

### How does the tool-call summarization reduce token costs?

When the number of stored tool calls exceeds the cutoff calculated by `compute_tool_call_cutoff`, Goose spawns background tasks via `maybe_summarize_tool_pairs` to execute `summarize_tool_call` on eligible pairs. Each task replaces the original tool call and response messages—often containing large JSON payloads—with a single short summary message, drastically reducing the token count sent to the LLM while preserving conversational context.