How to Use the Soup CLI Model Registry for Lineage Tracking

The Soup CLI includes a lightweight SQLite-backed model registry that records training runs, configurations, artifacts, and lineage DAGs linking derived models together via the RegistryStore class and soup registry subcommands.

Every machine learning pipeline needs reproducibility and provenance tracking. The Soup CLI model registry, implemented in MakazhanAlpamys/Soup, solves this with a minimal SQLite store that captures complete lineage from base models through fine-tuning, evaluation, and deployment. This guide covers the core architecture, Python API, and CLI workflows for model lineage tracking.

Core Registry Architecture

The registry centers on RegistryStore in src/soup_cli/registry/store.py. This class provides a context-manager interface that lazy-opens SQLite connections, ensures schema integrity, and hardens file permissions.

Registry Components

• RegistryStore – Central store class with with RegistryStore() as store: context manager (lines 198-206)
• push – Creates entries with hashed configs and auto-generated IDs like reg_YYYYMMDD_<hex> (lines 56-69)
• resolve – Normalizes references (entry ID, prefix, name:tag, or registry:// URI) to concrete IDs (lines 7-11)
• add_lineage – Records directed edges with cycle detection via _reaches_ancestor (lines 522-55)
• get_ancestors / get_descendants – BFS graph walks for lineage traversal (lines 62-88 and 20-46)
• add_artifact – Attaches files with kind validation against _VALID_KINDS (lines 63-78)

Lineage Data Model

The registry stores lineage as a DAG (directed acyclic graph):

Concept	Implementation
Nodes	Rows in registry_entries table
Edges	Rows in registry_lineage (child_id → parent_id with relation column)
Relations	forked_from, merged_from, evaluated_with, promoted_from

The add_lineage method enforces acyclicity: it rejects self-references and any edge that would create a cycle through _reaches_ancestor.

CLI Workflows for Model Lineage Tracking

Register a Training Run

After soup train completes, push the run to the registry:

soup registry push \
    --run-id run_202611_abc123 \
    --name llama31-chat \
    --tag v1 \
    --notes "initial release"

The push command hashes the configuration, stores optional data hashes, and returns a short unique ID like reg_20260601_xxxxxx.

Resolve Registry References

The resolve method accepts multiple reference formats:

# These all resolve to the same entry

soup registry show llama31-chat:v1          # name:tag shortcut

soup registry show reg_20260601_abc123      # full entry ID

soup registry show registry://llama31-chat   # URI form (latest tag)

If a prefix matches multiple entries, resolve raises AmbiguousRefError with the candidates listed.

Record Lineage Relationships

Track model provenance by declaring parent-child relationships:

soup registry add-lineage \
    --child-id reg_20260801_xyz789 \
    --parent-id reg_20260601_abc123 \
    --relation forked_from

Valid relations are strictly enforced: forked_from, merged_from, evaluated_with, or promoted_from. The command fails if the edge would create a cycle in the DAG.

Visualize Lineage History

View the complete provenance tree for a model name:

soup history llama31-chat

Output shows the DAG structure:

llama31-chat:v1 (reg_20260601_abc123)
└─ fine-tuned (reg_20260801_xyz789)  ← forked_from
   └─ distilled (reg_20260915_def456) ← merged_from

The history command internally calls get_ancestors and get_descendants to build this visualization.

Attach Artifacts to Entries

Link generated files (model binaries, BOMs, evaluation results) to registry entries:

soup registry attach-artifact \
    --entry-id reg_20260601_abc123 \
    --kind bom \
    --path ./my-model-bom.json

The add_artifact method validates that:

• The kind exists in _VALID_KINDS
• The file path is inside the cwd snapshot captured at RegistryStore construction

Promote Entries Across Stages

Move tags to mark production readiness:

soup registry promote reg_20260601_abc123 --tag prod

This adds the prod tag to the entry, enabling soup registry show prod to resolve the production version.

Remove Entries Safely

Delete with automatic cleanup:

soup registry delete reg_20260601_abc123 --yes

SQLite ON DELETE CASCADE removes associated artifacts, lineage edges, and tags automatically.

Python API for Custom Tooling

Import RegistryStore directly for CI pipelines or custom workflows:

from soup_cli.registry.store import RegistryStore

with RegistryStore() as store:
    # Create entry programmatically

    entry_id = store.push(
        name="my-model",
        tag="v2",
        base_model="llama31",
        task="sft",
        run_id="run_202611_abc123",
        config={"learning_rate": 1e-4, "epochs": 3},
    )
    
    # Establish lineage

    store.add_lineage(
        child_id=entry_id,
        parent_id="reg_20260601_abc123",
        relation="forked_from"
    )
    
    # Query ancestry

    ancestors = store.get_ancestors(entry_id)
    print("Ancestors:", [a["id"] for a in ancestors])
    
    # Query descendants with depth limit

    descendants = store.get_descendants(entry_id, depth=2)

The Python API mirrors CLI capabilities while enabling programmatic orchestration.

Key Source Files

File	Purpose
src/soup_cli/registry/store.py	SQLite schema, CRUD operations, lineage DAG logic, artifact handling
src/soup_cli/registry/attach.py	Branch snapshot attachment utilities
docs/commands.md	Complete CLI reference for soup registry commands
docs/adapters-and-governance.md	Registry integration with adapters and governance workflows

Summary

• Model lineage tracking in Soup CLI uses a SQLite-backed registry with automatic DAG enforcement
• Lineage edges require validated relations (forked_from, merged_from, evaluated_with, promoted_from) and pass cycle detection
• Reference resolution supports multiple formats (ID, prefix, name:tag, registry:// URI)
• Artifact attachment validates file kinds and path containment for security
• Both CLI and Python APIs provide full access for interactive use and automation

Frequently Asked Questions

What database does the Soup model registry use?

The registry uses SQLite as its default backend. The RegistryStore class in src/soup_cli/registry/store.py manages lazy connection opening, schema creation, and file permission hardening through its context manager interface.

How does Soup prevent circular lineage relationships?

Cycle detection runs automatically via the _reaches_ancestor method called inside add_lineage. Before inserting any edge, the code checks whether the proposed parent already reaches the proposed child through existing edges. If so, the operation raises an error preserving DAG integrity.

Can I migrate registry data between machines?

Yes, the registry is file-based. The SQLite database lives at a configurable path (default: .soup/registry.db). Copy this file along with referenced artifacts to migrate complete lineage history. Ensure artifact paths remain valid or use relative paths within the captured working directory snapshot.

What artifact kinds are supported in the registry?

Built-in kinds include model binaries, evaluation results, and BOMs. The add_artifact method validates against _VALID_KINDS defined in store.py. Common values are model, eval, bom, and checkpoint. Custom kinds can be added to the source code if needed for specialized workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}