cmf
CMF library helps to collect and store information associated with ML pipelines. It tracks the lineages for artifacts and executions of distributed AI pipelines. It provides API's to record and query the metadata associated with ML pipelines. The framework adopts a data first approach and all artifacts recorded in the framework are versioned and identified by the content hash.
Learn how to implement custom storage backends for CMF artifacts. This guide shows you how to create Python classes and register them to handle artifact pulls efficiently.
How to Handle Edge Cases When Artifacts Share URIs but Have Different Names in CMFLearn to handle CMF artifacts sharing URIs but having different names to prevent lineage errors and metadata drift. Understand unexpected framework behavior and resolve potential conflicts.
How to Use DVC Ingest to Convert Existing Pipelines to CMFConvert existing DVC pipelines to CMF using cmf dvc ingest. Migrate dvc.lock stages to MLMD executions with versioned artifacts for seamless tracking.
How to Implement Cross-Framework Model Logging (PyTorch & scikit-learn) with CMFImplement cross-framework model logging for PyTorch and scikit-learn using CMF. Unified metadata tracking for ML models without framework-specific logic. Learn more!
How to Migrate from SQLite to PostgreSQL for Production CMF DeploymentsMigrate CMF deployments from SQLite to PostgreSQL for production. Configure five POSTGRES environment variables to switch to a scalable PostgreSQL backend.
Data Slice Tracking in CMF: Analyzing Dataset Subsets with Full ProvenanceAnalyze dataset subsets with CMF data slice tracking. Get precise model performance insights on specific data segments with full lineage metadata and versioned provenance artifacts.
How to Debug "DVC not Configured Correctly" Initialization Errors in CMFDebug DVC not configured correctly initialization errors in HPE CMF. Learn how to fix DVC remote configuration issues and prevent metadata tracking failures. Get your CMF project running smoothly.
How Execution Reuse Works with `create_new_execution` in CMFLearn how CMF execution reuse works with create_new_execution. Set to False to reuse MLMD executions by name or True to create new ones.
How to Export CMF Lineage Data for External Visualization ToolsExport CMF lineage data using the CLI or REST API. Generate JSON files or query D3 compatible graphs for Neo4j, GraphViz, and custom visualization tools.
How Artifact Version Tracking Prevents Duplicate Dataset Logging in CMFLearn how CMF artifact version tracking prevents duplicate dataset logging using content-derived hashes and ML Metadata to ensure data integrity. Avoid redundant logging effectively.
How to Implement Fine-Grained Metric Logging per Training Step with CMFLog fine grained metrics per training step with CMF. Accumulate data in memory and commit to Parquet files with provenance.
How to Configure and Run the CMF Server with a PostgreSQL BackendConfigure and run the CMF server with a PostgreSQL backend by setting environment variables and using docker compose. Learn how to ensure database health checks before CMF initialization.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →