How the Caveman Safety Classifier Determines Which Payloads Skip Transformation

The Caveman safety classifier skips payload transformation when operating in record mode, when no matching compressor exists for the content type, when the compressor's safety class requires content recovery (CCR) but no store is configured, or when compression fails to yield a smaller payload.

The Caveman engine (JuliusBrussee/caveman) implements a fail-closed safety architecture that prevents data loss during compression. This system relies on a safety classifier to determine whether a payload should undergo transformation or remain untouched based on the compressor's safety class and runtime configuration.

Safety Class Registry and Compressor Properties

The safety classification system resides in engine/safety/safety.go. Each compressor advertises its safety class via the SafetyClass() method, which returns a value from the S0 to S4 registry. This registry maps classes to critical properties that determine transformation eligibility:

// engine/safety/safety.go
var registry = map[Class]Info{
    S0: {Class: S0, Name: "S0", ByteSafe: true,  RequiresCCR: false, Reversible: true},
    S1: {Class: S1, Name: "S1", ByteSafe: true,  RequiresCCR: false, Reversible: true},
    S2: {Class: S2, Name: "S2", ByteSafe: false, RequiresCCR: false, Reversible: true},
    S3: {Class: S3, Name: "S3", ByteSafe: false, RequiresCCR: false, Reversible: false},
    S4: {Class: S4, Name: "S4", ByteSafe: false, RequiresCCR: true,  Reversible: false},
}

• S0-S1: Byte-safe compressors that do not require CCR
• S2-S3: Potentially lossy but reversible (S2) or non-reversible (S3) without CCR requirements
• S4: Lossy compressors that strictly require CCR infrastructure

Pre-Compression Skip Conditions in engine.go

The Engine.Compress method in engine/engine.go (lines 84-102) implements a series of fail-closed checks that cause immediate payload skipping before any transformation occurs.

Mode-Based Bypass and Registry Validation

The engine first checks if the operation mode bypasses transformation entirely. If the mode is set to ModeRecord, the payload skips compression immediately. Subsequently, the engine validates compressor availability and safety class recognition:

// engine/engine.go – early-exit logic
if opts.Mode.normalized() == ModeRecord {
    return res, nil                     // record mode → skip
}
comp, ok := e.registry.For(ct)
if !ok { return res, nil }              // no compressor → skip
info, known := safety.Lookup(comp.SafetyClass())
if !known { return res, nil }           // unknown safety → skip

CCR Requirements for Lossy Compressors (S4)

For S4-class compressors, the safety classifier enforces an additional hard requirement. If info.RequiresCCR is true but the engine has no ccr.Store configured and the caller has not requested external recovery, the payload skips transformation to prevent irreversible data loss:

if info.RequiresCCR && e.store == nil && !opts.ExternalRecovery {
    return res, nil                     // S4 without CCR → skip
}

Post-Compression Size Validation

Even when pre-compression checks pass, the safety classifier performs post-compression validation to ensure efficiency. After the compressor executes, engine/engine.go validates that the transformation actually reduced payload size:

// engine/engine.go – size check
if after >= before || bytes.Equal(out, input) {
    return res, nil                     // not actually smaller → skip
}

This check ensures that even byte-safe compressors (S0-S1) do not emit payloads that would waste processing resources or tokens.

Practical Implementation Examples

The following examples demonstrate how the safety classifier behaves under different configurations:

// Example 1 – S4 compressor without CCR store skips transformation
engine := caveman.New(nil, nil)                 // no CCR store
payload := []byte(`{"large":"data","...":""}`) // JSON payload
opts := caveman.Options{Mode: caveman.ModeNormal}
result, err := engine.Compress(payload, opts)
// result.Output equals original payload because S4 requires CCR

// Example 2 – Querying compressor safety class directly
c := compressors.NewJSONStrategy(nil) // Returns S4 class
cls := c.SafetyClass()
info, _ := safety.Lookup(cls)
fmt.Printf("Class %s requires CCR: %v\n", info.Name, info.RequiresCCR)
// Output: Class S4 requires CCR: true

// Example 3 – S0 compressor with CCR store processes successfully
engine := caveman.New(ccrStore, nil)           // CCR store provided
payload := []byte(`{"metadata":"value"}`)    // Small metadata (S0)
opts := caveman.Options{Mode: caveman.ModeNormal}
res, _ := engine.Compress(payload, opts)
// Transformation applied if size reduction achieved

Summary

• Record mode (ModeRecord) forces all payloads to skip transformation
• Missing compressors or unknown safety classes trigger immediate skips to prevent undefined behavior
• S4 safety class requires a configured ccr.Store or external recovery option; otherwise, payloads skip
• Size validation discards transformations that fail to reduce payload size or produce byte-identical output
• The registry in engine/safety/safety.go provides the authoritative mapping of compressor capabilities to safety requirements

Frequently Asked Questions

What triggers a payload to skip transformation in Caveman's safety classifier?

A payload skips transformation when any of five conditions are met: the engine operates in record mode, no compressor exists for the content type, the compressor's safety class is unknown, an S4-class compressor lacks CCR infrastructure, or the compressed output is not smaller than the input.

Why does the S4 safety class require a CCR store?

The S4 class represents lossy, non-reversible compression according to the registry defined in engine/safety/safety.go. Because S4 transformations cannot recover the original payload without external storage, the safety classifier enforces the RequiresCCR property to prevent permanent data loss when no recovery mechanism exists.

How does record mode affect the safety classifier's behavior?

When Options.Mode is set to ModeRecord, the Engine.Compress method returns immediately with the original payload before evaluating safety classes or compressor availability. This mode effectively bypasses the entire classification system to preserve data for logging or audit purposes.

What happens if a compressor produces output larger than the input?

The post-comparison logic in engine/engine.go checks if after >= before and discards the transformation, returning the original payload instead. This ensures the engine never emits expanded payloads, maintaining efficiency regardless of the compressor's safety classification.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}