Data Slice Tracking in CMF: Analyzing Dataset Subsets with Full Provenance

Data slice tracking in CMF creates immutable, versioned subsets of datasets that are registered as first-class provenance artifacts, enabling precise analysis of model performance on specific data segments while maintaining complete lineage metadata.

Data slice tracking is a core capability of the hewlettpackard/cmf (Common Metadata Framework) library designed to help ML practitioners isolate and analyze specific subsets of training data. By creating named, queryable views of your dataset that capture custom metadata and execution context, you can perform targeted diagnostics on demographic segments, rare examples, or out-of-distribution data. This feature integrates directly with CMF's ML-metadata graph to provide immutable audit trails for every subset used in your pipeline.

What Is Data Slice Tracking in CMF?

In CMF, a data slice is a named, versioned collection of file paths representing a subset of a larger dataset. When you create a slice using Cmf.DataSlice, you are not duplicating raw data but rather establishing a curated view with associated metadata that lives within the provenance graph.

Each slice acts as a provenance artifact that links files to their DVC hashes, execution context, and custom properties such as demographic tags or quality flags. Once committed, the slice becomes an immutable output of its creating execution, allowing downstream queries to trace exactly which data segments contributed to specific model training runs or evaluation metrics.

How Data Slice Tracking Works Under the Hood

The implementation in cmflib/cmf.py orchestrates four critical operations when you work with data slices:

1. Artifact Registration – The commit() method registers a Dataslice node in the ML-metadata graph (Neo4j/MLMD), capturing the slice's hash, git commit, and human-readable name.

2. Execution Linkage – Through create_new_artifact_event_and_attribution, the slice is bound as an output of the current execution, establishing lineage for downstream impact analysis.

3. Parquet Persistence – Slice metadata is stored in a Parquet table (dataslice_df) located at <ARTIFACTS>/<execution-uuid>/dataslice/<slice-name>, containing file paths, DVC hashes, and custom properties.

4. Queryable Interface – The read_dataslice() method loads this Parquet data into a pandas DataFrame, enabling SQL-like filtering on custom properties without reloading the entire dataset.

Creating and Committing Data Slices

To create a tracked subset, initialize a Cmf writer, instantiate a DataSlice object, and populate it with file paths and custom metadata. The following example demonstrates creating a demographic slice for bias analysis:

from cmflib.cmf import Cmf
import random

# Initialize CMF with pipeline context

metawriter = Cmf(filepath="mlmd", pipeline_name="FairnessPipeline")
metawriter.create_context(pipeline_stage="DataPreparation")
metawriter.create_execution(execution_type="SliceCreation")

# Create the data slice

dataslice = metawriter.create_dataslice(name="young-eu-users")

# Add files with custom demographic properties

for _ in range(20):
    idx = random.randrange(1, 100)
    file_path = f"data/train/{idx}.txt"
    dataslice.add_data(
        path=file_path,
        custom_properties={"age_group": "18-25", "region": "EU", "quality_score": "high"}
    )

# Commit to metadata store (creates Parquet and graph nodes)

dataslice.commit()

During commit(), CMF calculates DVC hashes for each file via dvc_get_hash and writes the complete metadata table to the artifacts directory.

Reading and Analyzing Dataset Subsets

Once committed, you can load any slice into a pandas DataFrame for analysis using read_dataslice(). This enables you to compute performance metrics for specific subsets without reprocessing the entire dataset:

import pandas as pd

# Load the committed slice

df: pd.DataFrame = metawriter.read_dataslice(name="young-eu-users")

# Analyze distribution of custom properties

print(df[["age_group", "region"]].value_counts())

# Filter for specific analysis

eu_high_quality = df[(df["region"] == "EU") & (df["quality_score"] == "high")]
print(f"High-quality EU samples: {len(eu_high_quality)}")

The returned DataFrame includes the file path as the index, the DVC hash for provenance verification, and all custom properties supplied during add_data().

Updating Slice Metadata

While the underlying file list remains immutable, CMF supports updating custom properties for specific records using update_dataslice(). This is useful for correcting labels or adding post-hoc annotations without creating a new slice:

# Update demographic label for a specific file

metawriter.update_dataslice(
    name="young-eu-users",
    record="data/train/42.txt",
    custom_properties={"age_group": "26-35"}  # Corrected classification

)

# Verify the update

df = metawriter.read_dataslice(name="young-eu-users")
print(df.loc["data/train/42.txt", "age_group"])

This operation rewrites the Parquet file at <ARTIFACTS>/<execution-uuid>/dataslice/<slice-name> with the modified properties while preserving the original file hash and lineage.

Practical Applications of Data Slice Tracking

Data slice tracking enables several critical ML workflows that rely on subset analysis:

• Bias Detection – Isolate demographic sub-populations (age, gender, region) into separate slices to compare model accuracy and fairness metrics across groups.
• Out-of-Distribution Analysis – Create slices for edge cases or rare examples to verify model robustness on non-representative data.
• Regulatory Auditability – Maintain immutable records of exactly which data subsets were used for training versus validation, satisfying compliance requirements for explainable AI.

Because each slice is linked to its creating execution in the metadata graph, you can query relationships such as "show all slices produced by this training run" or "identify which model versions were trained on the biased demographic subset."

Summary

• Data slice tracking creates named, versioned subsets of datasets as first-class provenance artifacts in CMF's metadata graph.
• Slices are stored as Parquet tables containing file paths, DVC hashes, and custom properties, located under <ARTIFACTS>/<execution-uuid>/dataslice/<slice-name>.
• The Cmf.DataSlice class in cmflib/cmf.py provides methods for creating (create_dataslice), populating (add_data), persisting (commit), and reading (read_dataslice) subsets.
• Each committed slice is automatically linked to its execution context via create_new_artifact_event_and_attribution, enabling full lineage tracking.
• Subset analysis is performed by loading slices into pandas DataFrames, allowing targeted diagnostics on specific data segments without infrastructure overhead.

Frequently Asked Questions

How does data slice tracking differ from standard dataset versioning in CMF?

Standard dataset versioning tracks the entire folder or file collection as a single artifact, while data slice tracking creates queryable, named subsets with attached metadata. Slices reference specific files within a larger dataset and maintain their own custom properties, enabling granular analysis of demographics or edge cases without splitting your data into separate physical directories.

Where does CMF store data slice metadata?

Slice metadata is persisted in a Parquet table stored at <ARTIFACTS>/<execution-uuid>/dataslice/<slice-name> within your CMF workspace. The table contains columns for file paths, DVC hashes, and any custom properties provided to add_data(). Additionally, a Dataslice node is created in the ML-metadata graph (Neo4j/MLMD) to track provenance and execution lineage.

Can I modify a data slice after it has been committed?

You can update custom properties of existing records using update_dataslice(), which rewrites the Parquet file with new metadata values. However, the underlying file list and DVC hashes remain immutable to preserve provenance. To change the file composition, you must create a new slice with a distinct name or version.

How does data slice tracking help identify model bias?

By creating slices that isolate specific demographic segments (e.g., age_group="18-25", region="EU"), you can compute model performance metrics (accuracy, loss, fairness scores) for each subset individually. Because CMF links slices to model executions, you can compare how a model performs across different populations and detect disparities that aggregate metrics might obscure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}