libCacheSim Trace Formats: Complete Guide to CSV, VSCSI, Binary, and OracleGeneral Support

libCacheSim supports nine distinct trace formats—including CSV, VSCSI, binary, OracleGeneral, plain text, and its native LCS format—each defined in the trace_type_e enum and dispatched through dedicated parser functions in reader.c.

The libCacheSim library (available at 1a1a11a/libcachesim) provides a flexible trace ingestion layer designed to handle diverse storage workload formats. Whether you are analyzing block traces from cloud physics workloads or proprietary Oracle dumps, understanding the supported libCacheSim trace formats is essential for correctly initializing simulations and cache evaluations.

Supported Trace Format Types

libCacheSim categorizes traces using the trace_type_e enumeration defined in libCacheSim/include/libCacheSim/enum.h. The runtime dispatch logic resides in libCacheSim/traceReader/reader.c, where each format maps to a specific setup and read function.

CSV_TRACE

Comma-separated values with optional header support. This format expects delimited text files where columns represent request attributes such as object ID, size, and timestamp.

In reader.c, CSV traces initialize via csv_setup_reader and parse individual requests through csv_read_one_req (lines 151‑166). When using the C API, you must specify column mappings through the reader_init_param_t structure:

reader_init_param_t init = default_reader_init_params();
init.obj_id_field = 5;       // column index for object id
init.obj_size_field = 4;     // column index for size
init.time_field = 2;         // column index for timestamp
init.has_header = true;
init.delimiter = ',';

BIN_TRACE

Binary dump format containing raw request records. This format stores structured data as raw bytes rather than text, reducing file size and parsing overhead.

The parser initializes via binaryReader_setup and reads via binary_read_one_req (lines 62‑64 in reader.c). Binary traces offer faster ingestion speeds compared to text-based formats for large-scale simulations.

VSCSI_TRACE

The vscsi format originated from the VSCsi benchmark suite and remains common in storage research. It represents a specialized binary format for SCSI command traces.

Implementation resides in vscsiReader_setup and vscsi_read_one_req (lines 65‑68). The Node.js bindings expose this type under the short name "vscsi".

ORACLE_GENERAL_TRACE

OracleGeneral binary traces (typically named oracleGeneral.bin) represent a proprietary format used in Oracle database workload studies. This format requires specific byte-level parsing distinct from standard binary traces.

The library handles this through oracleGeneralBin_setup and oracleGeneralBin_read_one_req (lines 74‑86 in reader.c).

Additional Formats

• PLAIN_TXT_TRACE: Space or tab-delimited plain text processed by txt_read_one_req (lines 59‑62).
• LCS_TRACE: libCacheSim's native binary format for optimized reuse.
• TWR_TRACE / TWRNS_TRACE: Specialized formats for TWR and TWR-NS workloads.
• VALPIN_TRACE: Format used in Valpin paper research.
• UNKNOWN_TRACE: Fallback type for unsupported formats.

How to Select and Configure Trace Formats

libCacheSim offers two methods for specifying trace types: explicit API declaration or automatic file extension detection.

Explicit API Specification

When calling open_trace(), pass the appropriate trace_type_e value as the second argument:

#include "libCacheSim.h"

reader_t *r = open_trace("data/workload.csv", CSV_TRACE, &init_params);
request_t *req = new_request();

while (read_one_req(r, req) == 0) {
    // Process request
}

Valid enum values include CSV_TRACE, VSCSI_TRACE, ORACLE_GENERAL_TRACE, BIN_TRACE, and PLAIN_TXT_TRACE.

Automatic Detection

The utility function detect_trace_type_from_path() (implemented in cli_reader_utils.c) inspects file suffixes to determine format:

• .csv → CSV_TRACE
• .vscsi → VSCSI_TRACE
• .bin → BIN_TRACE or ORACLE_GENERAL_TRACE (context-dependent)

If detection fails or the file lacks a standard extension, you must provide the type explicitly to avoid falling back to UNKNOWN_TRACE.

Practical Code Examples

C API: Reading a CSV Trace

The following example from example/cacheSimulator/main.c demonstrates a complete simulation loop using CSV input:

#include "libCacheSim.h"

int main() {
    reader_init_param_t init = default_reader_init_params();
    init.obj_id_field = 5;
    init.obj_size_field = 4;
    init.time_field = 2;
    init.has_header = true;
    init.delimiter = ',';

    reader_t *r = open_trace("data/cloudPhysicsIO.csv", CSV_TRACE, &init);
    request_t *req = new_request();

    while (read_one_req(r, req) == 0) {
        printf("obj_id=%lu size=%lu time=%lu\n",
               req->obj_id, req->obj_size, req->clock_time);
    }

    free_request(req);
    close_reader(r);
    return 0;
}

Node.js: VSCSI Trace Simulation

The Node.js bindings expose supported types through getSupportedTraceTypes():

const cachesim = require('libCacheSim-node');

console.log('Supported formats:', cachesim.getSupportedTraceTypes());
// Output includes: 'vscsi', 'csv', 'txt', 'binary', 'oracle'

const result = cachesim.runSimulation(
  '../data/cloudPhysicsIO.vscsi',
  'vscsi',    // matches VSCSI_TRACE
  'lru',
  '2gb'
);

Key Implementation Files

Understanding the trace format architecture requires familiarity with these source files:

• libCacheSim/include/libCacheSim/enum.h: Declares trace_type_e and maps human-readable format names to integer constants.
• libCacheSim/traceReader/reader.c: Contains the switch statement dispatching to format-specific setup functions (lines 151‑176).
• libCacheSim-node/index.js: JavaScript interface exposing getSupportedTraceTypes() and the runSimulation() wrapper.
• cli_reader_utils.c: Implements automatic trace type detection from file paths.

Summary

• libCacheSim supports nine trace formats defined in enum.h: CSV, binary, plain text, VSCSI, OracleGeneral, LCS, TWR, TWR-NS, and Valpin.
• Format dispatch occurs in reader.c, with specific parsers like csv_read_one_req and vscsi_read_one_req handling ingestion.
• Use open_trace() with an explicit trace_type_e value or rely on detect_trace_type_from_path() for automatic selection based on file extensions.
• The Node.js bindings provide getSupportedTraceTypes() for JavaScript users, returning short names like "csv", "vscsi", and "oracle".
• Binary formats (BIN_TRACE, ORACLE_GENERAL_TRACE) offer superior performance for large-scale simulations compared to text-based alternatives.

Frequently Asked Questions

How do I determine which trace type constant to use for my file?

Check the file extension against the mappings in detect_trace_type_from_path() within cli_reader_utils.c. For .csv files use CSV_TRACE, for .vscsi use VSCSI_TRACE, and for Oracle binary dumps use ORACLE_GENERAL_TRACE. If your file lacks a standard extension, inspect the byte structure or refer to the source documentation in enum.h to match the format specification.

Can I use libCacheSim with custom trace formats not listed in the enum?

The library does not support arbitrary custom formats without source modification. You must either convert your data to one of the supported types (such as CSV or plain text) or implement a new parser following the pattern in reader.c and add a corresponding entry to trace_type_e in enum.h.

What is the performance difference between binary and CSV trace formats?

Binary formats like BIN_TRACE and ORACLE_GENERAL_TRACE parse significantly faster than CSV_TRACE because they avoid string-to-integer conversion and delimiter scanning. For large-scale cache simulations involving billions of requests, binary traces reduce I/O overhead and CPU usage during the warm-up phase.

Does the Node.js wrapper support all C API trace formats?

Yes, the Node.js bindings expose all major formats through getSupportedTraceTypes(), including "csv", "vscsi", "binary", "oracle", and "txt". However, fine-grained initialization parameters (such as CSV column mappings) may require using the C API directly or modifying the binding layer in libCacheSim-node/index.js.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}