# How Automatic Context Cutoff Works in ChocolateLMLite: Token and Time-Based Trimming

> Discover how ChocolateLMLite automatically cuts off context with token and time-based trimming before each LLM request. Optimize your conversation history efficiently.

- Repository: [Segment (gpsnmeajp)/chocolatelmlite](https://github.com/gpsnmeajp/chocolatelmlite)
- Tags: internals
- Published: 2026-03-02

---

**ChocolateLMLite automatically trims conversation history using a two-stage process that enforces token budgets and optional time windows before every LLM request.**

Managing long conversation histories in LLM applications requires careful token management to avoid exceeding model limits. In the open-source project **gpsnmeajp/chocolatelmlite**, the **automatic context cutoff** system ensures that each request stays within budget by intelligently pruning older messages. This mechanism combines token-based truncation with optional time-based filtering to maintain context relevance while preventing API errors.

## Two-Stage Context Trimming Pipeline

The cutoff operates in two coordinated stages managed by the `FileManager` and `Tokens` classes.

### Stage 1: Time-Based Cutoff (Optional)

If `TalkHistoryCutoffByPastHours` is greater than zero, the system removes entries older than the specified hour limit. When `TalkHistoryCutoffBeforeSummary` is enabled, the algorithm protects messages preceding the latest persona summary from removal. This logic resides in `FileManager.ApplyTalkHistoryTimeCutoff` (lines 1381–1413 in [`src/FileManager.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/FileManager.cs)).

### Stage 2: Token-Budget Cutoff

After time filtering, `Tokens.TrimTalkTokens` (lines 42–65 in [`src/Tokens.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/Tokens.cs)) processes the remaining history. Starting from the newest entry, it accumulates messages until the total token count—system prompt plus approximately 200 tokens of overhead—would exceed `TalkHistoryCutoffThreshold`. The resulting trimmed list becomes the final context sent to the LLM.

## Configuration Settings for Context Management

The cutoff behavior is controlled through `FileManager.GeneralSettings`:

- `TalkHistoryCutoffThreshold`: Hard token ceiling (default: 40,000 tokens)
- `TalkHistoryCutoffByPastHours`: Hours to retain (default: 0, disabled)
- `TalkHistoryCutoffBeforeSummary`: Protects pre-summary messages when true (default: false)

These settings are deserialized from the persona JSON definition in [`src/Persona.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/Persona.cs) (around line 749).

## Implementation Details and Code Flow

The LLM driver orchestrates the pipeline in [`src/LLM.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/LLM.cs) (line 198). It first applies the time-based filter, then invokes the token trimmer. If the final message count equals zero, the system aborts with an error indicating the token limit is too low.

Practical example of the flow:

```csharp
// Retrieve cached history
var allMessages = fileManager.GetAllTalkHistoryAllFromActivePersonaCached();

// Apply time-based filtering
int archivedByTime;
var timeFiltered = fileManager.GetTalkHistoryWithTimeCutoffFromActivePersona(
    activePersonaSettings.TalkHistoryCutoffByPastHours,
    activePersonaSettings.TalkHistoryCutoffBeforeSummary,
    activePersonaSummary?.Timestamp ?? 0,
    out archivedByTime);

// Enforce token budget
var trimmed = Tokens.TrimTalkTokens(
    systemPrompt,
    timeFiltered,
    fileManager.GeneralSettings.TalkHistoryCutoffThreshold);

```

Runtime configuration adjustments:

```csharp
// Increase budget to 80K tokens
fileManager.GeneralSettings.TalkHistoryCutoffThreshold = 80 * 1000;

// Enable 12-hour retention with summary protection
fileManager.GeneralSettings.TalkHistoryCutoffByPastHours = 12;
fileManager.GeneralSettings.TalkHistoryCutoffBeforeSummary = true;

```

## Error Handling When Context Is Fully Trimmed

When `TalkHistoryCutoffThreshold` is too restrictive, `Tokens.TrimTalkTokens` returns an empty collection. The LLM layer detects this condition and throws an `InvalidOperationException` with a message containing "会話履歴が全てカットオフ" (all conversation history cut off).

Handle this scenario as follows:

```csharp
try
{
    var reply = await llm.GetResponseAsync(...);
}
catch (InvalidOperationException ex) when (ex.Message.Contains("会話履歴が全てカットオフ"))
{
    // Prompt user to increase threshold or reduce system prompt
    Console.WriteLine("Context fully trimmed. Raise TalkHistoryCutoffThreshold or reduce system prompt size.");
}

```

## Summary

- ChocolateLMLite implements **automatic context cutoff** through two sequential stages: optional time-based filtering followed by mandatory token-budget trimming.
- The `FileManager.ApplyTalkHistoryTimeCutoff` method in [`src/FileManager.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/FileManager.cs) handles temporal pruning, while `Tokens.TrimTalkTokens` in [`src/Tokens.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/Tokens.cs) enforces the token ceiling.
- Configuration occurs via `TalkHistoryCutoffThreshold`, `TalkHistoryCutoffByPastHours`, and `TalkHistoryCutoffBeforeSummary` in `FileManager.GeneralSettings`.
- The LLM pipeline in [`src/LLM.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/LLM.cs) coordinates both stages and throws descriptive errors when the context is completely eliminated.

## Frequently Asked Questions

### What happens if the token limit is set too low in ChocolateLMLite?

If `TalkHistoryCutoffThreshold` is insufficient to accommodate even the system prompt plus overhead, `Tokens.TrimTalkTokens` returns an empty message list. The LLM driver in [`src/LLM.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/LLM.cs) (around line 204) detects this condition and raises an exception with the message "会話履歴が全てカットオフ", indicating that no conversation history remains for the model to process.

### How does the time-based cutoff interact with persona summaries?

When `TalkHistoryCutoffBeforeSummary` is enabled, the time-based cutoff algorithm in `FileManager.ApplyTalkHistoryTimeCutoff` compares entry timestamps against the latest summary timestamp. Messages created before the summary are retained regardless of the `TalkHistoryCutoffByPastHours` setting, ensuring that the persona's accumulated context remains intact.

### Can I disable the automatic context cutoff entirely?

You cannot completely disable token-based trimming, as it prevents API errors from exceeding model limits. However, you can effectively disable time-based filtering by setting `TalkHistoryCutoffByPastHours` to `0` (the default). To minimize truncation, set `TalkHistoryCutoffThreshold` to a high value appropriate for your LLM's context window.

### Where is the token counting logic implemented?

The core token estimation and trimming logic resides in [`src/Tokens.cs`](https://github.com/gpsnmeajp/chocolatelmlite/blob/main/src/Tokens.cs), specifically within the `TrimTalkTokens` method (lines 42–65). This method estimates token counts including approximately 200 tokens of overhead for tools and auto-inserted content, then truncates the conversation history from the oldest entries outward while preserving the newest messages.