# How to Implement Fine-Grained Metric Logging per Training Step with CMF

> Log fine grained metrics per training step with CMF. Accumulate data in memory and commit to Parquet files with provenance.

- Repository: [Hewlett Packard Enterprise/cmf](https://github.com/hewlettpackard/cmf)
- Tags: how-to-guide
- Published: 2026-03-03

---

**The CMF library provides a two-phase API where `log_metric()` accumulates step-level data in memory during training, and `commit_metrics()` serializes the series to versioned Parquet files with full DVC and MLMD provenance.**

The Hewlett Packard Enterprise **Collaborative Machine-Learning Framework (CMF)** enables detailed experiment tracking through its step-wise metric logging capabilities. Unlike frameworks that force immediate I/O on every step, CMF uses an in-memory buffering strategy that eliminates overhead in tight training loops while ensuring reproducible storage. This guide demonstrates how to use the `log_metric()` and `commit_metrics()` APIs from [`cmflib/cmf.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/cmf.py) to capture training metrics at every epoch or iteration.

## Understanding the CMF Metric Logging Architecture

CMF exposes two complementary methods in the `Cmf` class for **fine-grained metric logging**. The **accumulation phase** stores dictionaries in memory, while the **commit phase** handles serialization and provenance tracking.

### In-Memory Accumulation with log_metric

The `log_metric()` method (implemented in [`cmflib/cmf.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/cmf.py) at lines 1200–1226) accepts a `name` parameter and a dictionary of custom properties. Each call appends a new entry to `self.metrics[name]` using an incrementing sequence number as the key, preserving execution order without filesystem I/O.

### Persistent Storage with commit_metrics

When training completes, `commit_metrics()` (lines 1248–1290 in [`cmflib/cmf.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/cmf.py)) converts the accumulated dictionary into a **pandas DataFrame** via `DataFrame.from_dict(..., orient="index")`. It writes this to a **Parquet** file under the CMF artifacts directory hierarchy (`self.ARTIFACTS_PATH/.../metrics`), then registers the file with **DVC** using `commit_output` and captures the hash and URL. Finally, it creates a *Step_Metrics* artifact in the **ML Metadata (MLMD)** store, linking the metrics to the current execution and context.

## Implementing Fine-Grained Metric Logging

### Basic Per-Step Logging

Log metrics at each epoch without I/O overhead by accumulating data in memory:

```python
from cmflib.cmf import Cmf

# Initialize CMF context

metawriter = Cmf()

# Training loop

for epoch in range(1, 11):
    # Compute metrics

    train_loss = 0.42 - (epoch * 0.01)
    train_acc = 0.85 + (epoch * 0.01)
    
    # Accumulate in memory (no I/O)

    metawriter.log_metric(
        "training_metrics",
        {"epoch": epoch, "loss": train_loss, "accuracy": train_acc}
    )

# Persist after loop completion

metawriter.commit_metrics("training_metrics")

```

The `log_metric` calls populate `self.metrics["training_metrics"]` with sequence-indexed entries. As shown in the official example at [`examples/example-get-started/src/train.py`](https://github.com/hewlettpackard/cmf/blob/main/examples/example-get-started/src/train.py) (lines 58–73), this pattern separates high-frequency metric capture from storage operations.

### Distributed Training with Ray Tune

For hyperparameter tuning with Ray, use `CmfRayLogger` to maintain isolated contexts per trial:

```python
from cmflib.cmf_ray_logger import CmfRayLogger

def train_with_ray(config):
    trial_id = config["trial_id"]
    cmf_logger = CmfRayLogger(trial_id)
    
    for step in range(config["max_steps"]):
        loss = compute_loss(step)
        # Log per-step metrics

        cmf_logger.cmf_obj[trial_id].log_metric(
            metrics_name="ray_training_metrics",
            custom_properties={"step": step, "loss": loss}
        )
    
    # Commit trial metrics

    cmf_logger.cmf_obj[trial_id].commit_metrics("ray_training_metrics")

```

This approach, detailed in [`cmflib/cmf_ray_logger.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/cmf_ray_logger.py) (lines 66–78), maintains fine-grained metrics for parallel training runs while preserving DVC versioning and MLMD lineage per trial.

### Retrieving Historical Metrics

Access stored step metrics by loading the Parquet file back into a DataFrame:

```python
from cmflib.cmf import Cmf

cmf = Cmf()
metrics_df = cmf.read_metrics("training_metrics")
print(metrics_df.head())

```

The `read_metrics` method locates the Parquet file created by `commit_metrics` and returns the full step-wise history for analysis or visualization.

## Summary

- **Zero-overhead accumulation**: `log_metric()` stores step data in a Python dictionary during training loops, avoiding per-step I/O bottlenecks.
- **Atomic commits**: `commit_metrics()` serializes accumulated metrics to Parquet, registers with DVC for Git-tracked versioning, and creates MLMD artifacts for lineage.
- **Framework integration**: Use `CmfRayLogger` for automatic per-trial context isolation in Ray Tune distributed workflows.
- **Reproducible queries**: Retrieve historical metrics via `read_metrics()` as pandas DataFrames for downstream analysis.

## Frequently Asked Questions

### What is the performance impact of logging metrics every training step?

CMF's design ensures **zero filesystem I/O** during `log_metric()` calls, as data accumulates only in `self.metrics` memory. The expensive serialization, DVC hashing, and MLMD registration occur only once when `commit_metrics()` is called, typically after epoch completion or at training end.

### How does CMF handle metrics from distributed training runs?

The `CmfRayLogger` class in [`cmflib/cmf_ray_logger.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/cmf_ray_logger.py) creates isolated CMF contexts per Ray trial, allowing each worker to maintain independent metric buffers. Each trial calls `commit_metrics()` independently, producing separate versioned Parquet files with unique DVC hashes and MLMD execution links.

### Can I log non-scalar metrics like confusion matrices or embeddings?

The `log_metric()` API accepts any dictionary of custom properties, but `commit_metrics()` specifically converts the accumulated data to a pandas DataFrame for Parquet serialization. For complex artifacts like confusion matrices, use CMF's artifact logging APIs instead of the step-wise metric buffer.

### Where are the committed metric files stored?

Parquet files are written to the CMF artifacts directory hierarchy under `self.ARTIFACTS_PATH/.../metrics` as implemented in the `commit_metrics` method. DVC tracks these files via `commit_output`, and MLMD stores the file hash and URL as a *Step_Metrics* artifact property, enabling retrieval through `read_metrics()`.