How to Use the Trace Analyzer in libCacheSim for Performance Insights

To use the trace analyzer in libCacheSim for performance insights, build the traceAnalyzer binary, run it against your trace file with analysis flags such as --common, and visualize the generated .dat files using the Python plotting scripts bundled in the repository.

The trace analyzer in libCacheSim is a standalone binary built from the 1a1a11a/libcachesim repository that scans cache workload traces to compute locality statistics, reuse distributions, and popularity metrics. By processing raw trace data through modular analysis engines, you can extract quantitative insights about temporal locality and access patterns to optimize cache sizing and eviction policies.

Architecture of the Trace Analyzer

The analyzer is implemented in libCacheSim/traceAnalyzer/analyzer.h and analyzer.cpp around the TraceAnalyzer class, which orchestrates per-request processing and delegates to specialized modules.

Core Components

The architecture consists of four key parts:

• TraceAnalyzer class: The core driver defined in libCacheSim/traceAnalyzer/analyzer.h that reads traces record-by-record and forwards requests to enabled analysis modules.
• Analysis modules: Individual engines such as ReqRate, ReuseDistribution, SizeDistribution, and Popularity that implement the add_req(request_t*) method to update internal statistics.
• reader_t abstraction: The generic trace reader from libCacheSim/reader.h that parses formats like CSV or VSCSI and supplies request_t objects.
• CLI wrapper: The entry point in libCacheSim/bin/traceAnalyzer/main.cpp parses arguments, instantiates the analyzer, and outputs results.

Execution Flow

According to the source code in analyzer.cpp, the execution follows this sequence:

1. Initialization: The constructor validates warm-up parameters and allocates module objects based on the analysis_option_t struct.
2. Trace Processing: The run() method loops over the trace, normalizes timestamps to a relative base (start_ts_), maintains an obj_map_ for per-object metadata, and calls module->add_req(req) for each request.
3. Post-Processing: After consumption, post_processing() aggregates hit-count histograms and computes popularity rankings.
4. Output Generation: The gen_stat_str() method produces human-readable summaries, while each module's dump() method writes .dat files for visualization.

Building and Running the Analyzer

Build Instructions

Compile the tool from the repository root using CMake:

mkdir -p build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

The binary is produced at build/_build/bin/traceAnalyzer.

Command-Line Interface

Invoke the analyzer with the trace path, format, and analysis options:

./bin/traceAnalyzer PATH_TO_TRACE TRACE_TYPE [OPTIONS]

Key arguments include:

• --common: Enables core analyses including stat, reqRate, size, reuse, and popularity.
• --all: Runs every available module including experimental features.
• --accessPattern, --reqRate, --size, --reuse, --popularity: Fine-grained toggles for individual modules.
• -o <dir>: Specifies the output directory for .dat files (defaults to current directory).
• --num-req=N: Processes only the first N requests for quick tests.
• --warmup-sec=S: Excludes the first S seconds from statistics.

Example command to generate common statistics:

./bin/traceAnalyzer ../data/twitter_cluster52_10m.csv csv --common

This produces:

• stat: Concise textual summary printed to stdout.
• traceStat: Cumulative summary for multiple runs.
• *.dat files: Time-series data for each enabled module (e.g., twitter_cluster52_10m.size, twitter_cluster52_10m.reuse).

Visualizing Results

Use the Python scripts in scripts/traceAnalysis/ to generate plots:

python3 scripts/traceAnalysis/req_rate.py twitter_cluster52_10m.reqRate_w300
python3 scripts/traceAnalysis/size.py twitter_cluster52_10m.size
python3 scripts/traceAnalysis/reuse_heatmap.py twitter_cluster52_10m.reuseWindow_w300

These scripts output PNG/SVG files showing request-rate heatmaps, size distributions, and reuse patterns.

Programmatic Usage in C++

You can embed the analyzer directly in C++ applications using the public API:

#include "traceAnalyzer/analyzer.h"
#include "reader.h"

int main() {
    // Create a reader for a CSV trace
    reader_t *reader = create_reader("data/trace.csv", TRACE_TYPE_CSV, nullptr);

    // Configure analysis options
    traceAnalyzer::analysis_option_t opt = traceAnalyzer::default_option();
    opt.common = true;
    opt.reuse = true;

    // Set optional parameters
    traceAnalyzer::analysis_param_t param = traceAnalyzer::default_param();

    // Instantiate and run analyzer
    traceAnalyzer::TraceAnalyzer analyzer(reader, "output_dir", opt, param);
    analyzer.run();

    // Print summary
    std::cout << analyzer << std::endl;

    close_reader(reader);
    return 0;
}

The analysis_option_t struct mirrors CLI flags, allowing you to enable specific modules programmatically. The constructor handles initialization while run() executes the full analysis pipeline.

Interpreting Analyzer Output

The stat file generated by gen_stat_str() contains critical performance metrics:

number of requests: 10000000, number of objects: 897664
compulsory miss ratio (req/byte): 0.0898/0.0865
X-hit (number of obj accessed X times): 323699(0.3606), 218436(0.2433)...
freq (fraction) of the most popular obj: 546563(0.0547)...

Key metrics to analyze:

• Cold miss ratio: The fraction of distinct objects versus total requests; low values indicate strong temporal locality suitable for caching.
• X-hit histogram: Shows the distribution of objects accessed exactly X times. A steep drop-off after the first access suggests good cacheability.
• Popularity rank: The request frequency of the most popular objects. The slope of this distribution indicates Zipf-like skew, helping you select appropriate eviction policies.

The per-module .dat files provide time-series data revealing request-rate bursts for capacity planning, size-distribution trends for object memory layout, and reuse heatmaps showing temporal locality clusters.

Summary

• Build the traceAnalyzer binary from libCacheSim/bin/traceAnalyzer/ using CMake to produce the analysis tool.
• Run the analyzer with --common to generate core statistics including reuse distributions, size histograms, and popularity rankings.
• Process specific request counts using --num-req or exclude warm-up periods with --warmup-sec to focus on steady-state behavior.
• Visualize output using the Python scripts in scripts/traceAnalysis/ to identify temporal locality patterns and request spikes.
• Embed the TraceAnalyzer class directly in C++ applications by configuring analysis_option_t and calling run() for programmatic trace analysis.

Frequently Asked Questions

What trace formats does the libCacheSim trace analyzer support?

The analyzer supports multiple trace formats through the reader_t abstraction defined in libCacheSim/reader.h, including CSV, VSCSI, and other formats enumerated in trace_type_e. When invoking the binary, specify the format as the second argument (e.g., csv or vscsi) to enable the appropriate parser.

How do I enable specific analysis modules instead of running all of them?

Use the fine-grained boolean flags in the analysis_option_t struct or their CLI equivalents. For example, set opt.reuse = true in C++ or pass --reuse on the command line to enable only the reuse distribution analysis, rather than using the --common or --all bundles.

Can I integrate the trace analyzer into my own C++ application?

Yes, the TraceAnalyzer class in libCacheSim/traceAnalyzer/analyzer.h exposes a public API for embedding. Instantiate the class with a configured reader_t, output directory, and option flags, then call the run() method to execute the analysis pipeline without using the CLI binary.

What is the difference between the stat and traceStat output files?

The stat file contains a human-readable summary of the current trace analysis generated by gen_stat_str(), including object counts and miss ratios. The traceStat file serves as a cumulative log that appends results from multiple runs, useful for comparing statistics across different traces or configuration parameters over time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}