# How to Use the Soup CLI Model Registry for Lineage Tracking

> Learn to use the Soup CLI model registry for robust lineage tracking. Discover how to record training runs, configurations, artifacts, and link derived models with `soup registry` subcommands.

- Repository: [Alpamys Makazhan/Soup](https://github.com/MakazhanAlpamys/Soup)
- Tags: how-to-guide
- Published: 2026-09-06

---

**The Soup CLI includes a lightweight SQLite-backed model registry that records training runs, configurations, artifacts, and lineage DAGs linking derived models together via the `RegistryStore` class and `soup registry` subcommands.**

Every machine learning pipeline needs **reproducibility** and **provenance tracking**. The Soup CLI model registry, implemented in `MakazhanAlpamys/Soup`, solves this with a minimal SQLite store that captures complete lineage from base models through fine-tuning, evaluation, and deployment. This guide covers the core architecture, Python API, and CLI workflows for model lineage tracking.

## Core Registry Architecture

The registry centers on **`RegistryStore`** in [`src/soup_cli/registry/store.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/registry/store.py). This class provides a context-manager interface that lazy-opens SQLite connections, ensures schema integrity, and hardens file permissions.

### Registry Components

- **`RegistryStore`** – Central store class with `with RegistryStore() as store:` context manager (lines 198-206)
- **`push`** – Creates entries with hashed configs and auto-generated IDs like `reg_YYYYMMDD_<hex>` (lines 56-69)
- **`resolve`** – Normalizes references (entry ID, prefix, `name:tag`, or `registry://` URI) to concrete IDs (lines 7-11)
- **`add_lineage`** – Records directed edges with cycle detection via `_reaches_ancestor` (lines 522-55)
- **`get_ancestors` / `get_descendants`** – BFS graph walks for lineage traversal (lines 62-88 and 20-46)
- **`add_artifact`** – Attaches files with kind validation against `_VALID_KINDS` (lines 63-78)

### Lineage Data Model

The registry stores lineage as a **DAG** (directed acyclic graph):

| Concept | Implementation |
| --- | --- |
| **Nodes** | Rows in `registry_entries` table |
| **Edges** | Rows in `registry_lineage` (`child_id → parent_id` with `relation` column) |
| **Relations** | `forked_from`, `merged_from`, `evaluated_with`, `promoted_from` |

The `add_lineage` method enforces acyclicity: it rejects self-references and any edge that would create a cycle through `_reaches_ancestor`.

## CLI Workflows for Model Lineage Tracking

### Register a Training Run

After `soup train` completes, push the run to the registry:

```bash
soup registry push \
    --run-id run_202611_abc123 \
    --name llama31-chat \
    --tag v1 \
    --notes "initial release"

```

The `push` command hashes the configuration, stores optional data hashes, and returns a short unique ID like `reg_20260601_xxxxxx`.

### Resolve Registry References

The `resolve` method accepts multiple reference formats:

```bash

# These all resolve to the same entry

soup registry show llama31-chat:v1          # name:tag shortcut

soup registry show reg_20260601_abc123      # full entry ID

soup registry show registry://llama31-chat   # URI form (latest tag)

```

If a prefix matches multiple entries, `resolve` raises `AmbiguousRefError` with the candidates listed.

### Record Lineage Relationships

Track model provenance by declaring parent-child relationships:

```bash
soup registry add-lineage \
    --child-id reg_20260801_xyz789 \
    --parent-id reg_20260601_abc123 \
    --relation forked_from

```

Valid relations are strictly enforced: `forked_from`, `merged_from`, `evaluated_with`, or `promoted_from`. The command fails if the edge would create a cycle in the DAG.

### Visualize Lineage History

View the complete provenance tree for a model name:

```bash
soup history llama31-chat

```

Output shows the DAG structure:

```

llama31-chat:v1 (reg_20260601_abc123)
└─ fine-tuned (reg_20260801_xyz789)  ← forked_from
   └─ distilled (reg_20260915_def456) ← merged_from

```

The `history` command internally calls `get_ancestors` and `get_descendants` to build this visualization.

### Attach Artifacts to Entries

Link generated files (model binaries, BOMs, evaluation results) to registry entries:

```bash
soup registry attach-artifact \
    --entry-id reg_20260601_abc123 \
    --kind bom \
    --path ./my-model-bom.json

```

The `add_artifact` method validates that:
- The kind exists in `_VALID_KINDS`
- The file path is inside the cwd snapshot captured at `RegistryStore` construction

### Promote Entries Across Stages

Move tags to mark production readiness:

```bash
soup registry promote reg_20260601_abc123 --tag prod

```

This adds the `prod` tag to the entry, enabling `soup registry show prod` to resolve the production version.

### Remove Entries Safely

Delete with automatic cleanup:

```bash
soup registry delete reg_20260601_abc123 --yes

```

SQLite `ON DELETE CASCADE` removes associated artifacts, lineage edges, and tags automatically.

## Python API for Custom Tooling

Import `RegistryStore` directly for CI pipelines or custom workflows:

```python
from soup_cli.registry.store import RegistryStore

with RegistryStore() as store:
    # Create entry programmatically

    entry_id = store.push(
        name="my-model",
        tag="v2",
        base_model="llama31",
        task="sft",
        run_id="run_202611_abc123",
        config={"learning_rate": 1e-4, "epochs": 3},
    )
    
    # Establish lineage

    store.add_lineage(
        child_id=entry_id,
        parent_id="reg_20260601_abc123",
        relation="forked_from"
    )
    
    # Query ancestry

    ancestors = store.get_ancestors(entry_id)
    print("Ancestors:", [a["id"] for a in ancestors])
    
    # Query descendants with depth limit

    descendants = store.get_descendants(entry_id, depth=2)

```

The Python API mirrors CLI capabilities while enabling programmatic orchestration.

## Key Source Files

| File | Purpose |
| --- | --- |
| [`src/soup_cli/registry/store.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/registry/store.py) | SQLite schema, CRUD operations, lineage DAG logic, artifact handling |
| [`src/soup_cli/registry/attach.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/registry/attach.py) | Branch snapshot attachment utilities |
| [`docs/commands.md`](https://github.com/MakazhanAlpamys/Soup/blob/main/docs/commands.md) | Complete CLI reference for `soup registry` commands |
| [`docs/adapters-and-governance.md`](https://github.com/MakazhanAlpamys/Soup/blob/main/docs/adapters-and-governance.md) | Registry integration with adapters and governance workflows |

## Summary

- **Model lineage tracking in Soup CLI** uses a SQLite-backed registry with automatic DAG enforcement
- **Lineage edges** require validated relations (`forked_from`, `merged_from`, `evaluated_with`, `promoted_from`) and pass cycle detection
- **Reference resolution** supports multiple formats (ID, prefix, `name:tag`, `registry://` URI)
- **Artifact attachment** validates file kinds and path containment for security
- **Both CLI and Python APIs** provide full access for interactive use and automation

## Frequently Asked Questions

### What database does the Soup model registry use?

**The registry uses SQLite** as its default backend. The `RegistryStore` class in [`src/soup_cli/registry/store.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/src/soup_cli/registry/store.py) manages lazy connection opening, schema creation, and file permission hardening through its context manager interface.

### How does Soup prevent circular lineage relationships?

**Cycle detection runs automatically** via the `_reaches_ancestor` method called inside `add_lineage`. Before inserting any edge, the code checks whether the proposed parent already reaches the proposed child through existing edges. If so, the operation raises an error preserving DAG integrity.

### Can I migrate registry data between machines?

**Yes, the registry is file-based.** The SQLite database lives at a configurable path (default: `.soup/registry.db`). Copy this file along with referenced artifacts to migrate complete lineage history. Ensure artifact paths remain valid or use relative paths within the captured working directory snapshot.

### What artifact kinds are supported in the registry?

**Built-in kinds include model binaries, evaluation results, and BOMs.** The `add_artifact` method validates against `_VALID_KINDS` defined in [`store.py`](https://github.com/MakazhanAlpamys/Soup/blob/main/store.py). Common values are `model`, `eval`, `bom`, and `checkpoint`. Custom kinds can be added to the source code if needed for specialized workflows.