# How Continue Handles Token Counting and Context Window Management

> Learn how Continue manages token counting and context window limits. It prunes content efficiently to stay within model constraints while preserving key information.

- Repository: [Continue/continue](https://github.com/continuedev/continue)
- Tags: internals
- Published: 2026-06-24

---

**Continue stays within model limits by counting tokens for every prompt component using model-specific encoders, then pruning the oldest or least-critical content until the total fits inside the context window while preserving system prompts, tool definitions, and recent exchanges.**

Continue, the open-source AI coding assistant, implements sophisticated token counting and context window management to ensure requests never exceed a model's maximum capacity. According to the continuedev/continue source code, the system combines accurate token encoding with intelligent pruning strategies to maximize available context while dynamically adjusting for each model's specific tokenizer and safety requirements.

## Token Counting Implementation in [`core/llm/countTokens.ts`](https://github.com/continuedev/continue/blob/main/core/llm/countTokens.ts)

The core token accounting logic resides in **[`core/llm/countTokens.ts`](https://github.com/continuedev/continue/blob/main/core/llm/countTokens.ts)**, which provides synchronous and asynchronous methods for calculating token costs across different content types.

### Synchronous and Async Token Counting

The **`countTokens(content, modelName)`** function synchronously calculates tokens for `MessageContent` (strings or arrays of parts). It selects the appropriate encoder—`js-tiktoken` for OpenAI models or a Llama-based tokenizer for others—and passes text through `encoding.encode()`. After obtaining the base count, it applies model-specific scaling via `getAdjustedTokenCountFromModel`:

```typescript
return getAdjustedTokenCountFromModel(baseTokens, modelName);

```

For web-worker environments, **`countTokensAsync(content, modelName)`** returns a `Promise` using the `LlamaAsyncEncoder`:

```typescript
return (await encoding.encode(content ?? "")).length;

```

### Tool and Message Token Accounting

When calculating costs for function-calling capabilities, **`countToolsTokens(tools, modelName)`** computes the token overhead for tool names, descriptions, and parameter schemas, adding a fixed overhead of 12 tokens per tool:

```typescript
return numTokens + 12;

```

For complete chat messages, **`countChatMessageTokens(modelName, chatMessage)`** aggregates tokens for content, role tags, tool-call markers, and optional fields like `thinking` or `signature`. It starts with a `BASE_TOKENS` constant, then adds the encoded content length:

```typescript
let tokens = BASE_TOKENS;
// ...
tokens += countTokens(chatMessage.content, modelName);

```

### Safety Buffer Calculation

To prevent edge cases where trimming cuts too close to the limit, **`getTokenCountingBufferSafety(contextLength)`** reserves either 1,000 tokens or 2% of the context length, whichever is smaller:

```typescript
return Math.min(MAX_TOKEN_SAFETY_BUFFER, contextLength * TOKEN_SAFETY_PROPORTION);

```

## Context Window Pruning Strategies

When a prompt exceeds the model's context length, Continue employs a hierarchy of pruning helpers in **[`core/llm/countTokens.ts`](https://github.com/continuedev/continue/blob/main/core/llm/countTokens.ts)** to remove content strategically without breaking code structure or losing critical context.

### String-Based Pruning

**`pruneStringFromTop`** and **`pruneStringFromBottom`** remove tokens from the start or end of raw strings by encoding the full text, slicing the token array to `maxTokens`, then decoding the result. These functions handle unstructured text where line boundaries don't matter.

### Line-Based Pruning

**`pruneLinesFromTop`** and **`pruneLinesFromBottom`** preserve code integrity by pruning whole lines. They pre-compute token counts per line using `countTokens(line, modelName)`, then drop lines until the sum plus newline tokens is ≤ `maxTokens`. The VS Code extension uses these in [`extensions/vscode/src/apply/ApplyManager.ts`](https://github.com/continuedev/continue/blob/main/extensions/vscode/src/apply/ApplyManager.ts) and [`extensions/vscode/src/diff/vertical/manager.ts`](https://github.com/continuedev/continue/blob/main/extensions/vscode/src/diff/vertical/manager.ts) to fit diff payloads into the model window without splitting code blocks.

### High-Level Pruning Orchestration

**`pruneRawPromptFromTop`** orchestrates the full calculation by determining the *max usable tokens* (`contextLength - tokensForCompletion - safetyBuffer`), then invoking `pruneStringFromTop`:

```typescript
return pruneStringFromTop(modelName, maxTokens, prompt);

```

The autocomplete system leverages this in [`core/autocomplete/util/HelperVars.ts`](https://github.com/continuedev/continue/blob/main/core/autocomplete/util/HelperVars.ts) and [`core/autocomplete/templating/formatOpenedFilesContext.ts`](https://github.com/continuedev/continue/blob/main/core/autocomplete/templating/formatOpenedFilesContext.ts) via `pruneStringFromBottom` to retain only the most recent parts of large files.

## End-to-End Context Management Flow

When Continue builds an LLM request, it executes a deterministic pipeline to maximize context utilization:

1. **Collect data**: Aggregate system prompts, tool definitions, recent chat messages, and file snippets.
2. **Count tokens**: Calculate costs for each component using `countTokens` and `countChatMessageTokens`.
3. **Calculate usable space**:
   ```typescript
   const usableTokens = contextLength
                       - tokensForCompletion
                       - getTokenCountingBufferSafety(contextLength);
   ```

4. **Prune content**: Remove oldest parts (typically previous assistant messages) using `pruneRawPromptFromTop` until the total ≤ `usableTokens`.
5. **Log and send**: The LLM wrapper in **[`core/llm/index.ts`](https://github.com/continuedev/continue/blob/main/core/llm/index.ts)** records `promptTokens` and `generatedTokens` for telemetry before transmitting the trimmed payload.

The Next-Edit provider in [`core/nextEdit/providers/BaseNextEditProvider.ts`](https://github.com/continuedev/continue/blob/main/core/nextEdit/providers/BaseNextEditProvider.ts) additionally uses `countTokens` on individual lines to determine if file edits fit within remaining context.

## Practical Implementation Examples

### Counting Tokens for a Chat Message

```typescript
import { countChatMessageTokens } from "core/llm/countTokens.js";

const msg = {
  role: "assistant",
  content: "Here is the refactored code you asked for.",
  toolCalls: [{ name: "search", arguments: { query: "token counting" } }],
};

const tokenCount = countChatMessageTokens("gpt-4", msg);
console.log(`Message uses ${tokenCount} tokens`);

```

This example calls the function defined at lines 84-100 in [`core/llm/countTokens.ts`](https://github.com/continuedev/continue/blob/main/core/llm/countTokens.ts), which accounts for role tags, content, and tool call overhead.

### Pruning a Large Code Snippet for an 8K Window

```typescript
import { pruneStringFromBottom, getTokenCountingBufferSafety } from "core/llm/countTokens.js";

const fullCode = await readFile("src/largeFile.ts");
const contextLength = 8192;
const reservedForCompletion = 512;

const maxPromptTokens =
  contextLength - reservedForCompletion - getTokenCountingBufferSafety(contextLength);

const trimmed = pruneStringFromBottom("gpt-4", maxPromptTokens, fullCode);
console.log(`Trimmed prompt length: ${trimmed.length}`);

```

This approach ensures the model has guaranteed space for its response while keeping the most recent code context intact.

### Using High-Level Pruning in a VS Code Extension

```typescript
import { pruneRawPromptFromTop } from "core/llm/countTokens.js";

const prompt = await buildFullPrompt(); // Contains system msg, tools, file diff, etc.
const contextLength = 16384;            // gpt-4-turbo limit
const completionTokens = 1024;

const safePrompt = pruneRawPromptFromTop("gpt-4-turbo", contextLength, prompt, completionTokens);
sendToLLM(safePrompt);

```

The `pruneRawPromptFromTop` helper (lines 78-90) automatically calculates the safety buffer and trims from the beginning of the prompt, preserving the most recent conversational turns at the end.

## Summary

- **Centralized token counting** in [`core/llm/countTokens.ts`](https://github.com/continuedev/continue/blob/main/core/llm/countTokens.ts) supports both OpenAI (`js-tiktoken`) and Llama-based tokenizers with sync and async variants.
- **Model-specific adjustments** apply via `getAdjustedTokenCountFromModel`, while `getTokenCountingBufferSafety` reserves 2% or 1,000 tokens (whichever is smaller) to prevent boundary violations.
- **Strategic pruning** operates at string levels (unstructured text) and line levels (code preservation), with `pruneRawPromptFromTop` handling the final orchestration.
- **Integration across features** extends from autocomplete ([`HelperVars.ts`](https://github.com/continuedev/continue/blob/main/HelperVars.ts)) to VS Code diff management ([`ApplyManager.ts`](https://github.com/continuedev/continue/blob/main/ApplyManager.ts)) and Next-Edit providers, ensuring consistent context window compliance.
- **Telemetry verification** occurs in [`core/llm/index.ts`](https://github.com/continuedev/continue/blob/main/core/llm/index.ts), logging actual `promptTokens` and `generatedTokens` to confirm pruning succeeded.

## Frequently Asked Questions

### How does Continue count tokens for different models?

Continue selects the appropriate encoder based on the model name. OpenAI models use `js-tiktoken`, while other models use a Llama-based tokenizer. The `countTokens` function in [`core/llm/countTokens.ts`](https://github.com/continuedev/continue/blob/main/core/llm/countTokens.ts) automatically dispatches to the correct encoder and applies model-specific adjustments via `getAdjustedTokenCountFromModel` to account for different tokenization schemes.

### What is the token safety buffer and why is it needed?

The safety buffer reserves either 1,000 tokens or 2% of the total context length (whichever is smaller) through `getTokenCountingBufferSafety`. This prevents Continue from cutting too close to the model's absolute limit, accounting for potential tokenization discrepancies between the client's counter and the model's actual tokenizer, and ensuring room for metadata or formatting overhead.

### How does Continue decide what to prune from the context?

Continue prioritizes preserving system prompts, tool definitions, and the most recent user-assistant exchange. When pruning is necessary, it removes from the oldest parts first using `pruneRawPromptFromTop` or trims large file contents from the bottom (oldest lines) using `pruneStringFromBottom`. This strategy maintains conversational coherence while maximizing relevant recent context.

### Can I configure the context window size manually?

While Continue automatically detects context lengths from model configurations, you can influence behavior by selecting specific model presets or configuring `contextLength` in your Continue configuration file. The system will respect these limits when calculating `usableTokens` and apply the standard safety buffer formula to your specified window size.