Ossie dbt MSI Conversion Issues: Handling Lossy Transformations Between Formats

Converting between Apache Ossie and dbt MetricFlow Semantic Interface (MSI) formats results in four specific information-loss scenarios when moving from MSI to Ossie, while Ossie-to-MSI conversion requires heuristic mappings that may alter semantic granularity.

The apache/ossie repository provides the ossie-dbt converter to translate between Ossie's vendor-neutral YAML schema and dbt's JSON-based MSI manifest. Because Ossie intentionally maintains a lightweight core specification compared to dbt's expressive metric definitions, transformations inevitably encounter compatibility gaps that manifest as dropped elements or semantic approximations.

Understanding the Schema Architecture

The conversion system follows a hub-and-spoke architecture documented in docs/index.md, positioning Ossie as the central neutral format that vendor-specific converters must map into and out of. This design means the ossie-dbt converter must reconcile dbt's rich JSON manifest—supporting conversion funnels, visibility modifiers, and window functions—with Ossie's simplified YAML structure that lacks these vendor-specific extensions. Consequently, transformations are inherently asymmetrical: MSI-to-Ossie conversion is lossy by specification, while Ossie-to-MSI conversion applies best-effort heuristics.

Lossy Conversion from dbt MSI to Ossie

When processing target/semantic_manifest.json through the MSIToOSIConverter class in converters/dbt/src/ossie_dbt/msi_to_osi.py, the system detects unsupported features and emits ConverterIssue objects defined in converters/dbt/src/ossie_dbt/converter_issues.py. The following four issue types are recorded to stderr during conversion:

Conversion Funnel Metrics Dropped

CONVERSION_METRIC_DROPPED issues occur because Ossie's core specification has no equivalent metric type for dbt's conversion funnels. When the converter encounters metrics with conversion_funnel attributes, it omits them entirely from the output YAML and logs a warning referencing the metric name.

Private Metrics Dropped

PRIVATE_METRIC_DROPPED results from Ossie's lack of visibility modifiers. dbt's MSI allows visibility: private to hide metrics from downstream consumers, but Ossie does not expose this concept. The converter strips these metrics without round-trip preservation.

Natural Key Entities Dropped

NATURAL_ENTITY_DROPPED warnings indicate that natural-key entity types have been removed. Because Ossie lacks a natural-key entity type abstraction, any such entities defined in the dbt manifest are dropped during translation.

Cumulative Semantics Loss

CUMULATIVE_SEMANTICS_LOSS represents the most significant semantic degradation. dbt's MSI can encode window functions and grain specifications directly within metric expressions, but Ossie expressions are limited to simple arithmetic. The converter preserves only the base aggregation (e.g., SUM) while discarding window semantics, altering the metric's analytical behavior.

Best-Effort Conversion from Ossie to dbt MSI

The reverse direction implemented in converters/dbt/src/ossie_dbt/osi_to_msi.py uses the OSIToMSIConverter class to map Ossie YAML into dbt's JSON manifest. While this conversion does not drop elements, it makes unavoidable design decisions that may differ from original dbt semantics:

• Single aggregations like SUM(col) or COUNT(DISTINCT col) become SIMPLE metrics with metric_aggregation_params.
• Ratio expressions written as (expr_a) / (expr_b) are transformed into RATIO metrics with auto-generated sub-metrics.
• Complex expressions not matching simple aggregations or ratios are stored as SIMPLE metrics with the raw expression preserved verbatim.
• Time dimensions always receive TimeGranularity.DAY because Ossie does not store granularity metadata, forcing a default that may not match the original specification.

Detecting and Handling Conversion Issues

To identify information loss during MSI-to-Ossie conversion, capture the ConverterIssue objects returned by the Python API or monitor stderr when using the CLI:

# Convert dbt MSI → Ossie YAML (lossy)

ossie-dbt msi-to-osi -i target/semantic_manifest.json -o semantic_model.yaml

# warnings about ConverterIssueTypes will appear on stderr

# Convert Ossie YAML → dbt MSI (best‑effort)

ossie-dbt osi-to-msi -i semantic_model.yaml -o semantic_manifest.json

# resulting manifest can be loaded by MetricFlow

For programmatic access, inspect the issues attribute on the conversion result:

from ossie_dbt import MSIToOSIConverter, OSIToMSIConverter
from metricflow_semantics.model.dbt_manifest_parser import parse_manifest_from_dbt_generated_manifest
from pathlib import Path
import yaml

# MSI → Ossie with issue detection

manifest = parse_manifest_from_dbt_generated_manifest(Path("target/semantic_manifest.json").read_text())
result = MSIToOSIConverter().convert(manifest, osi_model_name="my_project")
for issue in result.issues:
    print(f"[warning] {issue.issue_type.value}: {issue.element_name}")

# Ossie → MSI

from ossie import OSIDocument
document = OSIDocument.model_validate(yaml.safe_load(Path("semantic_model.yaml").read_text()))
msi_result = OSIToMSIConverter().convert(document)
print(msi_result.output.model_dump_json(indent=2))

Summary

• Directional asymmetry exists between formats: MSI-to-Ossie loses data, while Ossie-to-MSI approximates semantics.
• Four specific loss types are defined in converter_issues.py: conversion metrics, private metrics, natural entities, and cumulative semantics.
• Time granularity defaults to daily precision when exporting to MSI due to Ossie's lack of granularity metadata.
• Window functions and visibility modifiers have no Ossie equivalent and are stripped during MSI import.
• CLI warnings provide actionable feedback via stderr, while the Python API exposes structured ConverterIssue objects for programmatic handling.

Frequently Asked Questions

Can I round-trip convert between Ossie and dbt MSI without losing data?

No, round-trip conversion is not lossless. Converting dbt MSI to Ossie and back to MSI will result in missing conversion funnels, lost visibility modifiers, and simplified cumulative metrics. The hub-and-spoke architecture documented in docs/index.md establishes Ossie as a least-common-denominator format, making perfect bidirectional translation impossible when dbt-specific features are present.

How do I handle ConverterIssue warnings during MSI to Ossie conversion?

Parse the issues list returned by MSIToOSIConverter().convert() or capture stderr output when using the ossie-dbt msi-to-osi CLI command. Each ConverterIssue object contains an issue_type (such as CONVERSION_METRIC_DROPPED or CUMULATIVE_SEMANTICS_LOSS) and the element_name that triggered the warning, allowing you to audit exactly which metrics or entities require manual reconstruction in Ossie.

Why are time dimensions always set to DAY granularity in MSI exports?

Because Ossie's core specification does not store time granularity metadata, the OSIToMSIConverter in osi_to_msi.py defaults all time dimensions to TimeGranularity.DAY. This heuristic ensures compatibility with MetricFlow but may require manual adjustment in the generated semantic_manifest.json if your original dbt specification used hourly, weekly, or monthly grains.

What happens to dbt ratio metrics when converting to Ossie?

Ratio metrics are preserved as arithmetic expressions (e.g., (numerator) / (denominator)) in Ossie's expression fields, but they lose their semantic identity as ratio-type metrics. When converting back to MSI, the OSIToMSIConverter reconstructs them as RATIO metrics with auto-generated sub-metrics, though the original sub-metric names and configurations may differ from the source.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}