# How TokenJuice Compression Middleware Functions in the OpenHuman Agent Turn Path

> Discover how TokenJuice compression middleware optimizes the OpenHuman agent turn path by compressing model outputs before client delivery and caching originals for retrieval.

- Repository: [Tiny Humans/openhuman](https://github.com/tinyhumansai/openhuman)
- Tags: internals
- Published: 2026-08-30

---

**The TokenJuice compression middleware intercepts model outputs in the TinyAgents pipeline to apply configurable lossy compression before payloads reach the client, preserving originals in a local content cache for later retrieval.**

TokenJuice is the OpenHuman subsystem responsible for shrinking large language model outputs through context-aware compression. Integrated into the TinyAgents middleware stack, this **TokenJuice compression middleware** evaluates every agent turn against user-defined profiles to determine whether to compress content before transmission.

## TokenJuice Compression Middleware Pipeline Integration

The compression logic is encapsulated in `ContextCompressionMiddleware`, which the TinyAgents harness inserts into the middleware stack during session initialization. According to the source code in [`openhuman/agent/tinyagents/mod_part_02.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/agent/tinyagents/mod_part_02.rs) and [`openhuman/agent/tinyagents/mod_part_03.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/agent/tinyagents/mod_part_03.rs), the middleware is positioned after standard tool-output and budget middlewares but before final response serialization.

This placement ensures that the middleware operates on the complete model-generated payload immediately after inference completes but before cost accounting and client delivery occur. The middleware receives the raw turn context and consults the active compression profile to decide whether to invoke the TokenJuice engine.

## Configuration Profiles and Compression Hints

The middleware’s behavior is governed by the `AgentTokenjuiceCompression` enum defined in [`openhuman/inference/tokenjuice/types.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/inference/tokenjuice/types.rs). This enum defines four distinct compression strategies:

- **Off** – Disables compression entirely.
- **Light** – Compresses only when the upstream model provides a positive compression hint.
- **Auto** – Follows the model’s compression hint (default behavior).
- **Full** – Forces compression on all eligible payloads regardless of hints.

The mapping from profile to execution logic occurs in [`openhuman/agent/tinyagents/host/budget_gate.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/agent/tinyagents/host/budget_gate.rs) at lines 220–225:

```rust
// Mapping of profile to hint (excerpt)
match self.compression {
    AgentTokenjuiceCompression::Off => CompressionHint::None,
    AgentTokenjuiceCompression::Light => match hint { ... },
    AgentTokenjuiceCompression::Auto | AgentTokenjuiceCompression::Full => hint,
}

```

This logic translates the high-level profile into a concrete `CompressionHint` that the middleware uses to gate the expensive compression operation.

## TokenJuice Compression Execution Flow

When the active profile permits compression, the `ContextCompressionMiddleware` delegates to the TokenJuice host module. In [`openhuman/modules/tokenjuice_host.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/modules/tokenjuice_host.rs) at line 23, the middleware invokes:

```rust
// TokenJuice host installs the compressor (excerpt)
crate::openhuman::inference::tokenjuice::ml::compress(&text, &options)

```

The underlying implementation in [`openhuman/inference/tokenjuice/ml/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/inference/tokenjuice/ml/mod.rs) performs the actual lossy reduction of the text payload. If the TokenJuice ML module is unavailable or fails to load, the system gracefully degrades by returning the original text unchanged, ensuring the agent turn path remains functional even when compression services are offline.

## Content Cache and Recovery Mechanism

Upon successful compression, the original uncompressed text is persisted to the workspace directory at `.<workspace>/.tokenjuice/ccr` (Content Cache Repository). This storage strategy enables deterministic recovery of the full payload without re-running inference.

The retrieval capability is exposed through the `tokenjuice_retrieve` tool implemented in [`openhuman/inference/tokenjuice/tools.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/inference/tokenjuice/tools.rs). Agents or client applications can invoke this tool to fetch the original text using the hash returned during the compression step.

## Practical Implementation Examples

To enable TokenJuice compression when building a harness session:

```rust
use openhuman::inference::tokenjuice::AgentTokenjuiceCompression;
use openhuman::agent::harness::session::builder::SessionBuilder;

// Build a session with auto compression (the default)
let session = SessionBuilder::new()
    .tokenjuice_compression(AgentTokenjuiceCompression::Auto)
    .build()
    .await?;

```

To explicitly disable compression for debugging or low-latency scenarios:

```rust
let session = SessionBuilder::new()
    .tokenjuice_compression(AgentTokenjuiceCompression::Off)
    .build()
    .await?;

```

To recover a compressed payload later in the conversation:

```rust
use openhuman::inference::tokenjuice::retrieve;

// `hash` is the token returned by the compression step
let original_text = retrieve(hash.to_string(), None).await?;

```

## Summary

- The **TokenJuice compression middleware** (`ContextCompressionMiddleware`) is integrated into the TinyAgents middleware stack between model inference and client response delivery.
- Compression behavior is controlled by the `AgentTokenjuiceCompression` enum, mapped to execution hints in [`budget_gate.rs`](https://github.com/tinyhumansai/openhuman/blob/main/budget_gate.rs) (lines 220–225).
- Active compression invokes `ml::compress` in the TokenJuice host module, with implementation details in [`openhuman/inference/tokenjuice/ml/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/inference/tokenjuice/ml/mod.rs).
- Original texts are archived to `.<workspace>/.tokenjuice/ccr` and recoverable via the `tokenjuice_retrieve` tool.
- Configuration occurs at session build time through `SessionBuilder`, supporting granular control from full compression to complete disablement.

## Frequently Asked Questions

### Where exactly does the TokenJuice middleware sit in the agent turn lifecycle?

The middleware is positioned in the TinyAgents pipeline immediately after model output generation and budget accounting, but before the final RPC response or UI rendering phase. This ensures compression overhead does not affect inference latency while minimizing payload sizes for transmission.

### What is the difference between the Auto and Light compression profiles?

**Auto** follows the model’s explicit compression hint without additional gating, compressing whenever the upstream system suggests it is beneficial. **Light** adds a conditional check that only compresses when the hint indicates a high-confidence opportunity for size reduction, preserving bandwidth for marginal cases.

### How does the system handle TokenJuice ML module failures?

If the ML module referenced in [`openhuman/inference/tokenjuice/ml/mod.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/inference/tokenjuice/ml/mod.rs) fails to load or crashes during compression, the `ml::compress` function returns the original text unchanged. This fail-safe behavior, implemented in the host module at [`openhuman/modules/tokenjuice_host.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/modules/tokenjuice_host.rs), guarantees agent availability even when compression services are degraded.

### Can I retrieve the original uncompressed text after it has been compressed?

Yes. Every compressed payload stores its original form in the workspace’s `.tokenjuice/ccr` directory. You can recover the full text using the `tokenjuice_retrieve` tool defined in [`openhuman/inference/tokenjuice/tools.rs`](https://github.com/tinyhumansai/openhuman/blob/main/openhuman/inference/tokenjuice/tools.rs), passing the unique hash identifier returned during the initial compression call.