# How TokenJuice Compression Reduces Tool Output Tokens by Up to 80% in OpenHuman

> Discover how TokenJuice compression slashes tool output tokens by 80% in OpenHuman. Learn how this model-driven side-car creates compact semantic representations for efficient agent turns.

- Repository: [Tiny Humans/openhuman](https://github.com/tinyhumansai/openhuman)
- Tags: deep-dive
- Published: 2026-08-27

---

**TokenJuice compression reduces agent tool output tokens by up to 80 % using a model‑driven Python side‑car that rewrites large payloads into compact semantic representations, caching the originals for on‑demand retrieval via the `tokenjuice_retrieve` tool.**

OpenHuman’s **TokenJuice** (often shortened to **tokenjuice**) is a lightweight compressor that intercepts tool outputs before they reach the LLM. When an agent’s tool produces a large textual payload, the core system hands the raw output to the tokenjuice service, which runs a small language model to shrink the content while preserving meaning. This design pattern directly lowers token usage costs and prevents context window overflow during agent turns.

## The Six‑Step Compression Pipeline

The TokenJuice compression flow operates as a coordinated pipeline across multiple Rust modules. Each step is designed to minimize latency while maximizing compression ratios.

**Step 1: Configuration Loading**
The core reads the `[tokenjuice]` configuration block from the workspace config, defined in [`src/openhuman/config/schema/tokenjuice.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/config/schema/tokenjuice.rs). Key parameters include `ml_compression_enabled`, `ml_target_ratio`, and `ml_max_input_chars`.

**Step 2: Host Invocation**
When a tool response exceeds size thresholds, the *tokenjuice host* in [`src/openhuman/modules/tokenjuice_host.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/modules/tokenjuice_host.rs) calls `crate::openhuman::inference::tokenjuice::ml::compress(&text, &options)`. This entry point bridges the synchronous tool execution flow with the asynchronous compression service.

**Step 3: Python Side‑car Execution**
The `ml::compress` function spawns a short‑lived Python process coordinated by [`src/openhuman/runtime/python_server/kompress.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/runtime/python_server/kompress.rs). This module loads the configured model (`ml_model_id`) and executes the `compress` routine, rewriting the input to meet the `ml_target_ratio` (default 0.25) while respecting `ml_max_input_chars`.

**Step 4: Caching and Token Generation**
The compressed result is stored in the TokenJuice cache ([`src/openhuman/inference/tokenjuice/cache.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/cache.rs)). A short hash token is generated and returned to the core, which replaces the original payload with a `{tokenjuice}` marker of minimal token count.

**Step 5: Retrieval Tool**
If the model later requires the full original text, it invokes the built‑in `tokenjuice_retrieve` tool implemented in [`src/openhuman/inference/tokenjuice/tools.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/tools.rs). The tool reads the cached entry using the hash token and streams the uncompressed content back to the model.

**Step 6: Savings Tracking**
Every compression event updates [`tokenjuice_savings.json`](https://github.com/tinyhumansai/openhuman/blob/main/tokenjuice_savings.json) via [`src/openhuman/inference/tokenjuice/savings.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/savings.rs). Empirical measurements in the test suite ([`tests/raw_coverage/..._raw_coverage_e2e.rs`](https://github.com/tinyhumansai/openhuman/blob/main/tests/raw_coverage/..._raw_coverage_e2e.rs)) demonstrate reductions from roughly 6 k tokens to 1.2 k tokens, achieving the advertised **up‑to‑80 % token reduction** per turn.

## Configuration and Tuning

Fine‑tuning TokenJuice compression behavior starts with the configuration schema in [`src/openhuman/config/schema/tokenjuice.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/config/schema/tokenjuice.rs). The system exposes several levers to balance compression aggressiveness against information fidelity.

- **`ml_compression_enabled`**: Boolean gate that disables the entire pipeline when set to `false`, causing the core to bypass the compressor and send raw outputs unchanged.
- **`ml_target_ratio`**: Controls the desired tokens‑to‑character ratio (default 0.25). Lowering this value increases compression but risks semantic loss.
- **`ml_max_input_chars`**: Sets the character threshold that triggers compression; content below this limit passes through uncompressed.
- **`ml_model_id`**: Specifies the distilled model (e.g., a distilled GPT‑2 variant) used by the Python side‑car for rewriting.

## Core Implementation Architecture

The TokenJuice subsystem spans seven primary source files that implement distinct responsibilities:

**[`src/openhuman/modules/tokenjuice_host.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/modules/tokenjuice_host.rs)**
Acts as the core entry point. This module determines when compression is required based on payload size and delegates to the inference layer.

**[`src/openhuman/runtime/python_server/kompress.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/runtime/python_server/kompress.rs)**
Orchestrates the Python side‑car process lifecycle. It handles model loading, process spawning, and inter‑process communication for the compression task.

**[`src/openhuman/inference/tokenjuice/ml.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/ml.rs)**
Provides the Rust wrapper around the Python process. This file defines the `compress` API signature and manages the serialization of text and options across the language boundary.

**[`src/openhuman/inference/tokenjuice/cache.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/cache.rs)**
Implements the storage layer for compressed payloads. It generates deterministic hash tokens and manages cache eviction policies to prevent memory bloat.

**[`src/openhuman/inference/tokenjuice/tools.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/tools.rs)**
Contains the `tokenjuice_retrieve` tool definition. This module allows agents to recover full original text on demand by querying the cache with a hash token.

**[`src/openhuman/inference/tokenjuice/savings.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/savings.rs)**
Persists token‑saving statistics to disk. This data powers dashboards and telemetry showing cumulative compression benefits across agent sessions.

## Enabling and Using TokenJuice Compression

### Enable Automatic Compression

To enable automatic compression for all qualifying tool outputs in an orchestrator turn, configure the builder with `AgentTokenjuiceCompression::Auto`:

```rust
use openhuman_core::openhuman::inference::tokenjuice::AgentTokenjuiceCompression;

let orchestrator = OrchestratorBuilder::new()
    .tokenjuice_compression(AgentTokenjuiceCompression::Auto)
    .build()
    .await?;

```

The orchestrator now invokes the TokenJuice host for every tool output that exceeds `ml_max_input_chars`.

### Retrieve Original Payloads Manually

Models or downstream tools can recover the full uncompressed text using the `tokenjuice_retrieve` tool:

```rust
use openhuman_core::openhuman::inference::tokenjuice::TokenjuiceRetrieveTool;

let retrieve = TokenjuiceRetrieveTool::new();
let result = retrieve
    .run(json!({ "token": "a1b2c3d4" }))
    .await?;
println!("Full original text: {}", result.output);

```

The hash token (`a1b2c3d4` in this example) corresponds to the compressed entry stored in the cache.

### Monitor Token Savings via RPC

Frontend applications can query compression statistics through the RPC layer:

```typescript
const stats = await coreRpcClient.call("openhuman.tokenjuice_savings_stats", {});
console.log(`Saved ${stats.saved_tokens} tokens (${stats.savings_percent}% reduction)`);

```

This integration allows real‑time display of session‑level token efficiency metrics.

## Summary

- **TokenJuice compression** replaces large tool outputs with compact semantic representations, reducing agent turn tokens by up to 80 %.
- The pipeline is implemented across [`src/openhuman/config/schema/tokenjuice.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/config/schema/tokenjuice.rs), [`tokenjuice_host.rs`](https://github.com/tinyhumansai/openhuman/blob/main/tokenjuice_host.rs), and the Python side‑car in [`kompress.rs`](https://github.com/tinyhumansai/openhuman/blob/main/kompress.rs).
- Original content remains accessible through the `tokenjuice_retrieve` tool defined in [`src/openhuman/inference/tokenjuice/tools.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/tools.rs).
- Compression aggressiveness is controlled via `ml_target_ratio`, while `ml_max_input_chars` determines the activation threshold.
- All savings are persisted to [`tokenjuice_savings.json`](https://github.com/tinyhumansai/openhuman/blob/main/tokenjuice_savings.json) via [`src/openhuman/inference/tokenjuice/savings.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/savings.rs) for telemetry and cost tracking.

## Frequently Asked Questions

### How does TokenJuice compression achieve an 80 % reduction in tool output tokens?

TokenJuice employs a **model‑driven rewrite** strategy rather than naive truncation. When a tool returns a large payload (e.g., 6 k tokens), the Python side‑car loads a distilled language model and generates a semantically equivalent paraphrase that retains critical information while minimizing token count. Empirical tests in the OpenHuman repository measured outputs compressed to approximately 1.2 k tokens, yielding an 80 % reduction before the LLM processes the turn.

### What happens to the original tool output after compression?

The original text is never discarded. It is stored in the **TokenJuice cache** ([`src/openhuman/inference/tokenjuice/cache.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/inference/tokenjuice/cache.rs)), indexed by a short hash token. The LLM receives only a `{tokenjuice}` marker and the compact representation. If the model requires the full text later in the conversation, it can invoke the `tokenjuice_retrieve` tool to stream the uncached original from disk.

### Can TokenJuice compression be disabled for specific agent turns?

Yes. The system respects the `ml_compression_enabled` boolean in the `[tokenjuice]` configuration block. When set to `false`, the core bypasses the compressor entirely, sending raw tool outputs directly to the model. Additionally, content shorter than `ml_max_input_chars` automatically skips compression, ensuring small payloads incur no overhead.

### Which machine learning model powers the TokenJuice side‑car?

The compression service uses a configurable **distilled language model** specified by the `ml_model_id` parameter. According to the source analysis, the default configuration typically references a distilled GPT‑2 variant or similar lightweight model capable of running efficiently within a short‑lived Python process spawned by [`src/openhuman/runtime/python_server/kompress.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/runtime/python_server/kompress.rs).