# How to Interpret libCacheSim Trace Analysis Output to Identify Cache Performance Issues

> Learn to interpret libCacheSim trace analysis output and diagnose cache performance issues. Decode metrics from stat files to pinpoint problems like cold misses and write amplification.

- Repository: [Juncheng Yang/libcachesim](https://github.com/1a1a11a/libcachesim)
- Tags: how-to-guide
- Published: 2026-02-23

---

**libCacheSim's traceAnalyzer condenses workload characteristics into a human-readable stat file, enabling you to diagnose issues like low temporal locality, write amplification, and popularity skew by decoding metrics such as cold-miss ratios and X-hit distributions.**

The libCacheSim trace analyzer is the primary diagnostic tool for understanding cache workload behavior. When you interpret libCacheSim trace analysis output, you are essentially translating a compressed summary of request patterns, object sizes, and access frequencies into actionable insights about cache configuration mismatches. This guide walks through every section of the analyzer's stat output, referencing the exact source locations in [`libCacheSim/traceAnalyzer/analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/analyzer.cpp) and related files where these metrics are computed.

## Header and Data Set Identification

The first line of the stat output identifies the source trace:

```

dat: <path-to-trace>

```

This field is generated in [`libCacheSim/traceAnalyzer/analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/analyzer.cpp) at lines 298–299, where the `reader_->trace_path` field is printed. **Verify that the path matches your intended workload**—a block-cache trace versus a key-value trace will exhibit radically different optimal cache policies.

## Basic Volume Metrics

The next block quantifies the overall workload scale:

```

number of requests: <n_req>, number of objects: <obj_map_.size()>
number of req GiB: <sum_obj_size_req / GiB>, number of obj GiB: <sum_obj_size_obj / GiB>

```

*Generated in* [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 298–303.

- **`n_req`** – Total trace length.
- **`obj_map_.size()`** – Count of distinct objects referenced.
- **GiB values** – Total volume of requests versus unique object volume.

**Diagnostic clues:**
- *Very few objects relative to requests* indicates high reuse, but may signal a pathological cache-size mismatch if the working set is tiny.
- *Huge request-GiB versus object-GiB* suggests many small objects, which may cause excessive metadata overhead.

## Cold-Miss (Compulsory Miss) Ratios

Cold misses represent the theoretical minimum miss rate—accesses to objects never seen before:

```

compulsory miss ratio (req/byte): <cold_miss_ratio>/<byte_cold_miss_ratio>

```

*Generated in* [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 304–306.

- **`cold_miss_ratio`** = `obj_map_.size() / n_req` – Fraction of unique object accesses.
- **`byte_cold_miss_ratio`** = `sum_obj_size_obj / sum_obj_size_req` – Byte-weighted version.

**Issue indicators:**
- Values **> 0.5** indicate *low reuse*—most accesses are first-time, meaning any cache will suffer high misses regardless of size.
- A large gap between request-ratio and byte-ratio signals many tiny objects, creating metadata-heavy workloads.

## Object Size Weighting

This metric reveals whether large objects dominate traffic:

```

object size weighted by req/obj: <mean_obj_size_req>/<mean_obj_size_obj>

```

*Generated in* [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 306–308.

- **`mean_obj_size_req`** – Average size per request (size-weighted by request count).
- **`mean_obj_size_obj`** – Average size per distinct object.

**Diagnostic insight:**
If `mean_obj_size_req` is significantly larger than `mean_obj_size_obj`, the workload contains a few large objects that dominate traffic. Consider **tiered caching** to isolate these heavy hitters.

## Frequency Mean

The frequency mean indicates average reuse intensity:

```

frequency mean: <freq_mean>

```

*Generated in* [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) line 310.

`freq_mean = n_req / obj_map_.size()` – Average accesses per object.

**Interpretation:**
- **Low (< 2)** indicates low reuse, typical of "write-only" or "append-only" workloads.
- **High (> 10)** indicates strong temporal locality; you can size caches aggressively with confidence.

## Time Span

The temporal coverage of the trace affects validity:

```

time span: <time_span> (<days> day)

```

*Generated in* [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 311–313.

**Warning signs:**
Very short spans containing millions of requests suggest *synthetic* or *compressed* traces. These may hide diurnal patterns that affect real production systems, leading to over-optimistic cache sizing.

## Operation Statistics

Operation breakdowns reveal write amplification risks:

```

op: SET:12345(0.12), GET:87654(0.88), ...
write: <n_write>(<ratio>), overwrite: <n_overwrite>(<ratio>), del: <n_del>(<ratio>)

```

*Generated by* `OpStat::operator<<` in [`traceAnalyzer/op.h`](https://github.com/1a1a11a/libcachesim/blob/main/traceAnalyzer/op.h) lines 29–48.

**Critical metrics:**
- **High write-to-read ratio** may cause write amplification; eviction policies must account for write cost.
- **Frequent overwrites** (`overwrite_cnt_`) can hide true miss rates because the same key is rewritten repeatedly, potentially masking cache inefficiency.

## X-Hit Distribution

The X-hit distribution shows reuse depth:

```

X-hit (number of obj accessed X times): 5720(0.0094), 4813(0.0079), …

```

*Generated in* [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 319–324.

`n_hit_cnt_[i]` records objects accessed exactly `i+1` times.

**Pattern recognition:**
- A *long tail* with many objects accessed only once signals low reuse.
- A *steep drop* after a few hits suggests a "heavy-head / light-tail" distribution where caching should focus on the most popular objects.

## Popularity Distribution

Popularity metrics reveal Zipfian characteristics:

```

freq (fraction) of the most popular obj: 74030(0.0173), 74007(0.0173), …

```

*Generated in* [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 326–331.

The `Popularity` class sorts objects by request count and stores the top-`track_n_popular_` frequencies.

**Cache sizing implications:**
- A **steep Zipf slope** (few objects capturing large fractions) means a small cache can achieve high hit rates.
- A **flat tail** (many objects with similar low frequencies) indicates need for larger caches or adaptive policies.

## Optional Advanced Statistics

The analyzer supports experimental detectors that appear only when enabled:

| Statistic | Source File | Diagnostic Value |
|-----------|-------------|------------------|
| **Size-change distribution** | [`traceAnalyzer/experimental/sizeChange.hpp`](https://github.com/1a1a11a/libcachesim/blob/main/traceAnalyzer/experimental/sizeChange.hpp) & `.cpp` | Tracks object growth/shrinkage for variable-size workloads (e.g., VMs). |
| **Scan detector** | [`traceAnalyzer/experimental/scanDetector.hpp`](https://github.com/1a1a11a/libcachesim/blob/main/traceAnalyzer/experimental/scanDetector.hpp) & `.cpp` | Identifies sequential scan phases that temporarily swamp caches. |
| **Popularity decay** | [`traceAnalyzer/popularityDecay.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/traceAnalyzer/popularityDecay.cpp) | Shows how object popularity evolves; fast decay implies need for recency-aware policies. |
| **Lifetime & future-reuse** | Various experimental files | Research-grade metrics for advanced algorithm design. |

These dump via `dump(output_path_)` calls in `TraceAnalyzer::run()` at lines 52–71 of [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp).

## Practical Code Examples

### Running the Analyzer and Capturing Output

```bash

# Build the binary first (see doc/quickstart_cachesim.md)

./bin/traceAnalyzer ../data/twitter_cluster52_10m.csv csv --common > twitter.stat

```

### Parsing Statistics with Python

Extract key metrics programmatically for CI pipelines:

```python
import re
from pathlib import Path

def parse_stat(file_path: Path) -> dict:
    """Return a dict with diagnostic fields from a traceAnalyzer stat file."""
    stats = {}
    with file_path.open() as f:
        txt = f.read()

    patterns = {
        "requests": r"number of requests: (\d+)",
        "objects": r"number of objects: (\d+)",
        "cold_miss_req": r"compulsory miss ratio \(req/byte\): ([\d.]+)/",
        "cold_miss_byte": r"compulsory miss ratio \(req/byte\): [\d.]+/([\d.]+)",
        "freq_mean": r"frequency mean: ([\d.]+)",
        "write_count": r"write: (\d+)\(",
        "overwrite_count": r"overwrite: (\d+)\(",
    }

    for key, pat in patterns.items():
        m = re.search(pat, txt)
        if m:
            stats[key] = m.group(1)

    return stats


if __name__ == "__main__":
    import json
    stat = parse_stat(Path("twitter.stat"))
    print(json.dumps(stat, indent=2))

```

### Normalizing Trace Filenames

Use the provided utility to maintain consistent naming conventions:

```python
from scripts.utils.trace_utils import extract_dataname

trace_path = "../data/twitter_cluster52_10m.csv"
dataname = extract_dataname(trace_path)
print(dataname)   # → twitter_cluster52_10m

```

This function strips common suffixes like `.csv`, `.zst`, and `_w300` to ensure generated plot files share a consistent base name, as implemented in [`scripts/utils/trace_utils.py`](https://github.com/1a1a11a/libcachesim/blob/main/scripts/utils/trace_utils.py) lines 14–44.

## Key Source Files

| File | Role |
|------|------|
| [`libCacheSim/traceAnalyzer/analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/analyzer.cpp) | Main driver; builds the textual stat output (lines 298–334). |
| [`libCacheSim/traceAnalyzer/op.h`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/op.h) | Formats per-operation statistics (`op`, `write`, `overwrite`, `del`) via `OpStat::operator<<` (lines 29–48). |
| [`libCacheSim/traceAnalyzer/popularity.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/popularity.cpp) | Computes the sorted frequency list for the popularity distribution section. |
| [`scripts/utils/trace_utils.py`](https://github.com/1a1a11a/libcachesim/blob/main/scripts/utils/trace_utils.py) | Helper utilities for trace filename normalization. |
| [`doc/quickstart_traceAnalyzer.md`](https://github.com/1a1a11a/libcachesim/blob/main/doc/quickstart_traceAnalyzer.md) | Official documentation for running the tool. |

## Summary

- **Cold-miss ratios** measure inherent reuse; values above 0.5 indicate low temporal locality that limits cache effectiveness regardless of size.
- **Size-weighted means** reveal whether tiny objects dominate metadata overhead or large objects dominate bandwidth.
- **X-hit and popularity distributions** expose the head-tail shape of the workload; steep Zipf curves allow small caches, while flat distributions require adaptive policies.
- **Operation statistics** flag write-heavy workloads and overwrite patterns that cause amplification or mask true miss rates.
- **Advanced detectors** (scan, popularity decay, size-change) provide research-grade insights for specialized workloads.

## Frequently Asked Questions

### What does a high compulsory miss ratio indicate in libCacheSim trace analysis?

A compulsory miss ratio greater than 0.5 indicates that more than half of all requests access objects never seen before in the trace. According to the calculation in [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 304–306 (`obj_map_.size() / n_req`), this signals low temporal locality. Even an infinitely large cache cannot reduce these misses, suggesting the workload is unsuitable for caching or requires prefetching strategies rather than eviction policy tuning.

### How can I identify if my workload has a Zipfian popularity distribution?

Examine the "freq (fraction) of the most popular obj" line in the stat output, generated in [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) lines 326–331. If the top few objects capture a large fraction of total requests (e.g., top 10 objects > 30% of traffic), the workload follows a steep Zipf curve. This pattern, stored by the `Popularity` class, indicates that a small cache can achieve high hit rates, and you should avoid over-provisioning cache space.

### What is the significance of the X-hit distribution in trace analysis?

The X-hit distribution, output at lines 319–324 of [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp), shows how many objects receive exactly X accesses. The `n_hit_cnt_[i]` array records objects accessed exactly `i+1` times. A long tail of single-access objects (high first value) confirms low reuse, while a steep drop after several hits reveals a "heavy-head" workload where caching should prioritize frequently re-accessed objects.

### When should I enable the advanced experimental statistics in traceAnalyzer?

Enable advanced detectors—such as the scan detector ([`scanDetector.hpp`](https://github.com/1a1a11a/libcachesim/blob/main/scanDetector.hpp)), size-change distribution ([`sizeChange.hpp`](https://github.com/1a1a11a/libcachesim/blob/main/sizeChange.hpp)), or popularity decay ([`popularityDecay.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/popularityDecay.cpp))—when the basic stats suggest anomalies you cannot explain. For example, if you see sporadic hit-rate collapses despite stable popularity, enable the scan detector to identify sequential read phases. These modules dump via `TraceAnalyzer::run()` at lines 52–71 of [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) only when explicitly compiled or flagged, making them suitable for research-grade analysis of variable-size objects (e.g., VM workloads) or rapidly shifting access patterns.