# How Execution Reuse Works with `create_new_execution` in CMF

> Learn how CMF execution reuse works with create_new_execution. Set to False to reuse MLMD executions by name or True to create new ones.

- Repository: [Hewlett Packard Enterprise/cmf](https://github.com/hewlettpackard/cmf)
- Tags: deep-dive
- Published: 2026-03-03

---

**Set `create_new_execution=False` in `Cmf.create_execution()` to reuse existing MLMD executions by name, or use the default `True` to always create fresh executions.**

The Hewlett Packard Enterprise CMF (Context Management Framework) library enables fine-grained control over ML Metadata (MLMD) execution creation through the `create_new_execution` parameter. When building reproducible machine learning pipelines with the `hewlettpackard/cmf` repository, understanding how this boolean flag governs execution reuse is essential for managing pipeline state and lineage across runs.

## Understanding the `create_new_execution` Flag

The `create_new_execution` parameter is a boolean flag that defaults to `True` in `Cmf.create_execution()`. This flag determines whether the library treats the `execution_type` argument as a generic type identifier or as a persistent name for lookup purposes. When set to `False`, the framework enables deterministic execution reuse by storing the type value as a custom `name` property, allowing subsequent calls to retrieve and update the same execution record.

## The Execution Creation Flow

The implementation spans two core files in `cmflib`. In [`cmflib/cmf.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/cmf.py), the `create_execution()` method (lines 61-102) receives the flag and delegates to `create_new_execution_in_existing_run_context()` (lines 384-398). This delegation passes control to [`cmflib/metadata_helper.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/metadata_helper.py), where `create_execution_with_type()` (lines 66-90) implements the branching logic:

- **`create_new_execution=True`** (default): The function always instantiates a brand-new `ml_metadata.proto.Execution` of the specified type. The execution carries no `name` property, making it impossible to retrieve later for reuse.

- **`create_new_execution=False`**: The function first queries existing executions for a matching `type_name` **and** `name` combination. If found, it returns the existing execution and updates its properties. If no match exists, it creates a new execution with the `name` property explicitly set to the `execution_type` value.

## Code Examples

### Default Behavior: Always Create Fresh Executions

When you omit the flag or pass `True`, every call generates a distinct execution without storage names.

```python
from cmflib.cmf import Cmf

cmf = Cmf(filepath="mlmd", pipeline_name="my_pipeline")
cmf.create_context(pipeline_stage="train")

# Each call creates a distinct execution

exec_1 = cmf.create_execution(
    execution_type="TrainingStep",
    custom_properties={"epoch": 1}
)  # Execution ID: 101

exec_2 = cmf.create_execution(
    execution_type="TrainingStep",
    custom_properties={"epoch": 2}
)  # Execution ID: 102 (new, not reused)

```

### Reuse Mode: Linking to Existing Executions

Set the flag to `False` to enable reuse. The `execution_type` becomes the execution's persistent name.

```python

# First call creates a named execution

exec_reuse = cmf.create_execution(
    execution_type="Preprocessing",
    custom_properties={"dataset_version": "v1.0"},
    create_new_execution=False
)  # Execution ID: 201, name="Preprocessing"

# Second call retrieves and updates the same execution

exec_same = cmf.create_execution(
    execution_type="Preprocessing",
    custom_properties={"dataset_version": "v1.1"},
    create_new_execution=False
)  # Execution ID: 201 (reused), properties updated

```

### Mixed Mode Limitations

Executions created with the default behavior can never be reused later, even if you subsequently pass `create_new_execution=False`.

```python

# Create execution without reuse capability

exec_no_name = cmf.create_execution("Evaluation")  # ID: 301

# Attempting to reuse fails because the original lacks a name property

exec_attempt = cmf.create_execution(
    execution_type="Evaluation",
    create_new_execution=False
)  # ID: 302 (new execution created, not reused)

```

## Summary

- The `create_new_execution` parameter in `Cmf.create_execution()` controls whether CMF creates fresh MLMD executions or reuses existing ones by name.
- **Default `True`**: Always creates new executions without storing a `name` property, preventing future reuse.
- **`False` mode**: Stores `execution_type` as the execution's `name` custom property, enabling lookup and reuse across pipeline runs.
- Reuse only works for executions originally created with `create_new_execution=False` according to the `hewlettpackard/cmf` source code in [`cmflib/metadata_helper.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/metadata_helper.py).
- Core logic resides in the `create_execution_with_type()` function, which queries existing executions by type and name when the flag is disabled.

## Frequently Asked Questions

### Can I reuse an execution that was created with the default `create_new_execution=True`?

No. Executions created with `create_new_execution=True` (the default) do not store the `name` custom property in MLMD. Since the reuse lookup in [`cmflib/metadata_helper.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/metadata_helper.py) specifically queries for executions matching both type and name, these executions remain invisible to the reuse mechanism. You must create the original execution with `create_new_execution=False` to enable future reuse.

### Where does CMF store the execution name for reuse lookups?

CMF stores the execution name as a **custom property** within the MLMD Execution proto. When `create_new_execution=False`, the `create_execution_with_type()` function in [`cmflib/metadata_helper.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/metadata_helper.py) explicitly sets the `name` property to the value of `execution_type`. This property serves as the lookup key when subsequent calls attempt to reuse the execution.

### What happens if I pass different `custom_properties` when reusing an execution?

The existing execution's properties are updated with the new values provided in the `custom_properties` dictionary. The function returns the same execution ID but modifies the execution's metadata to reflect the latest property values passed during the reuse call, as implemented in [`cmflib/metadata_helper.py`](https://github.com/hewlettpackard/cmf/blob/main/cmflib/metadata_helper.py) lines 80-90.

### Is it safe to use `create_new_execution=False` in concurrent pipeline runs?

Concurrent runs accessing the same MLMD backend with identical execution names and types will compete for the same execution record. The underlying MLMD store handles atomicity, but concurrent updates to custom properties may result in last-write-wins behavior. For isolated parallel runs, use distinct execution names or pipeline contexts rather than relying on reuse across concurrent processes.