# How to Use the Trace Analyzer in libCacheSim for Performance Insights

> Unlock performance insights with libCacheSim's trace analyzer. Build, run with flags, and visualize .dat files using bundled Python scripts.

- Repository: [Juncheng Yang/libcachesim](https://github.com/1a1a11a/libcachesim)
- Tags: how-to-guide
- Published: 2026-02-23

---

**To use the trace analyzer in libCacheSim for performance insights, build the `traceAnalyzer` binary, run it against your trace file with analysis flags such as `--common`, and visualize the generated `.dat` files using the Python plotting scripts bundled in the repository.**

The trace analyzer in libCacheSim is a standalone binary built from the `1a1a11a/libcachesim` repository that scans cache workload traces to compute locality statistics, reuse distributions, and popularity metrics. By processing raw trace data through modular analysis engines, you can extract quantitative insights about temporal locality and access patterns to optimize cache sizing and eviction policies.

## Architecture of the Trace Analyzer

The analyzer is implemented in [`libCacheSim/traceAnalyzer/analyzer.h`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/analyzer.h) and [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp) around the `TraceAnalyzer` class, which orchestrates per-request processing and delegates to specialized modules.

### Core Components

The architecture consists of four key parts:

- **`TraceAnalyzer` class**: The core driver defined in [`libCacheSim/traceAnalyzer/analyzer.h`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/analyzer.h) that reads traces record-by-record and forwards requests to enabled analysis modules.
- **Analysis modules**: Individual engines such as `ReqRate`, `ReuseDistribution`, `SizeDistribution`, and `Popularity` that implement the `add_req(request_t*)` method to update internal statistics.
- **`reader_t` abstraction**: The generic trace reader from [`libCacheSim/reader.h`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/reader.h) that parses formats like CSV or VSCSI and supplies `request_t` objects.
- **CLI wrapper**: The entry point in [`libCacheSim/bin/traceAnalyzer/main.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/bin/traceAnalyzer/main.cpp) parses arguments, instantiates the analyzer, and outputs results.

### Execution Flow

According to the source code in [`analyzer.cpp`](https://github.com/1a1a11a/libcachesim/blob/main/analyzer.cpp), the execution follows this sequence:

1. **Initialization**: The constructor validates warm-up parameters and allocates module objects based on the `analysis_option_t` struct.
2. **Trace Processing**: The `run()` method loops over the trace, normalizes timestamps to a relative base (`start_ts_`), maintains an `obj_map_` for per-object metadata, and calls `module->add_req(req)` for each request.
3. **Post-Processing**: After consumption, `post_processing()` aggregates hit-count histograms and computes popularity rankings.
4. **Output Generation**: The `gen_stat_str()` method produces human-readable summaries, while each module's `dump()` method writes `.dat` files for visualization.

## Building and Running the Analyzer

### Build Instructions

Compile the tool from the repository root using CMake:

```bash
mkdir -p build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

```

The binary is produced at `build/_build/bin/traceAnalyzer`.

### Command-Line Interface

Invoke the analyzer with the trace path, format, and analysis options:

```bash
./bin/traceAnalyzer PATH_TO_TRACE TRACE_TYPE [OPTIONS]

```

Key arguments include:

- `--common`: Enables core analyses including `stat`, `reqRate`, `size`, `reuse`, and `popularity`.
- `--all`: Runs every available module including experimental features.
- `--accessPattern`, `--reqRate`, `--size`, `--reuse`, `--popularity`: Fine-grained toggles for individual modules.
- `-o <dir>`: Specifies the output directory for `.dat` files (defaults to current directory).
- `--num-req=N`: Processes only the first N requests for quick tests.
- `--warmup-sec=S`: Excludes the first S seconds from statistics.

Example command to generate common statistics:

```bash
./bin/traceAnalyzer ../data/twitter_cluster52_10m.csv csv --common

```

This produces:

- `stat`: Concise textual summary printed to stdout.
- `traceStat`: Cumulative summary for multiple runs.
- `*.dat` files: Time-series data for each enabled module (e.g., `twitter_cluster52_10m.size`, `twitter_cluster52_10m.reuse`).

### Visualizing Results

Use the Python scripts in `scripts/traceAnalysis/` to generate plots:

```bash
python3 scripts/traceAnalysis/req_rate.py twitter_cluster52_10m.reqRate_w300
python3 scripts/traceAnalysis/size.py twitter_cluster52_10m.size
python3 scripts/traceAnalysis/reuse_heatmap.py twitter_cluster52_10m.reuseWindow_w300

```

These scripts output PNG/SVG files showing request-rate heatmaps, size distributions, and reuse patterns.

## Programmatic Usage in C++

You can embed the analyzer directly in C++ applications using the public API:

```cpp
#include "traceAnalyzer/analyzer.h"
#include "reader.h"

int main() {
    // Create a reader for a CSV trace
    reader_t *reader = create_reader("data/trace.csv", TRACE_TYPE_CSV, nullptr);

    // Configure analysis options
    traceAnalyzer::analysis_option_t opt = traceAnalyzer::default_option();
    opt.common = true;
    opt.reuse = true;

    // Set optional parameters
    traceAnalyzer::analysis_param_t param = traceAnalyzer::default_param();

    // Instantiate and run analyzer
    traceAnalyzer::TraceAnalyzer analyzer(reader, "output_dir", opt, param);
    analyzer.run();

    // Print summary
    std::cout << analyzer << std::endl;

    close_reader(reader);
    return 0;
}

```

The `analysis_option_t` struct mirrors CLI flags, allowing you to enable specific modules programmatically. The constructor handles initialization while `run()` executes the full analysis pipeline.

## Interpreting Analyzer Output

The `stat` file generated by `gen_stat_str()` contains critical performance metrics:

```

number of requests: 10000000, number of objects: 897664
compulsory miss ratio (req/byte): 0.0898/0.0865
X-hit (number of obj accessed X times): 323699(0.3606), 218436(0.2433)...
freq (fraction) of the most popular obj: 546563(0.0547)...

```

Key metrics to analyze:

- **Cold miss ratio**: The fraction of distinct objects versus total requests; low values indicate strong temporal locality suitable for caching.
- **X-hit histogram**: Shows the distribution of objects accessed exactly X times. A steep drop-off after the first access suggests good cacheability.
- **Popularity rank**: The request frequency of the most popular objects. The slope of this distribution indicates Zipf-like skew, helping you select appropriate eviction policies.

The per-module `.dat` files provide time-series data revealing request-rate bursts for capacity planning, size-distribution trends for object memory layout, and reuse heatmaps showing temporal locality clusters.

## Summary

- Build the `traceAnalyzer` binary from `libCacheSim/bin/traceAnalyzer/` using CMake to produce the analysis tool.
- Run the analyzer with `--common` to generate core statistics including reuse distributions, size histograms, and popularity rankings.
- Process specific request counts using `--num-req` or exclude warm-up periods with `--warmup-sec` to focus on steady-state behavior.
- Visualize output using the Python scripts in `scripts/traceAnalysis/` to identify temporal locality patterns and request spikes.
- Embed the `TraceAnalyzer` class directly in C++ applications by configuring `analysis_option_t` and calling `run()` for programmatic trace analysis.

## Frequently Asked Questions

### What trace formats does the libCacheSim trace analyzer support?

The analyzer supports multiple trace formats through the `reader_t` abstraction defined in [`libCacheSim/reader.h`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/reader.h), including CSV, VSCSI, and other formats enumerated in `trace_type_e`. When invoking the binary, specify the format as the second argument (e.g., `csv` or `vscsi`) to enable the appropriate parser.

### How do I enable specific analysis modules instead of running all of them?

Use the fine-grained boolean flags in the `analysis_option_t` struct or their CLI equivalents. For example, set `opt.reuse = true` in C++ or pass `--reuse` on the command line to enable only the reuse distribution analysis, rather than using the `--common` or `--all` bundles.

### Can I integrate the trace analyzer into my own C++ application?

Yes, the `TraceAnalyzer` class in [`libCacheSim/traceAnalyzer/analyzer.h`](https://github.com/1a1a11a/libcachesim/blob/main/libCacheSim/traceAnalyzer/analyzer.h) exposes a public API for embedding. Instantiate the class with a configured `reader_t`, output directory, and option flags, then call the `run()` method to execute the analysis pipeline without using the CLI binary.

### What is the difference between the `stat` and `traceStat` output files?

The `stat` file contains a human-readable summary of the current trace analysis generated by `gen_stat_str()`, including object counts and miss ratios. The `traceStat` file serves as a cumulative log that appends results from multiple runs, useful for comparing statistics across different traces or configuration parameters over time.