Implementing Unit Conversions within sdata Metadata Attributes: A Step-by-Step Guide

You can implement unit conversions in sdata by adding a convert_attribute method to the Metadata class in sdata/metadata.py that uses the Pint library to transform numeric values between compatible units while updating the attribute's unit string in-place.

The sdata library provides a structured way to store experimental and simulation data alongside rich metadata descriptions. While the existing Metadata class already extracts and stores unit strings via helpers like set_unit_from_name, it lacks built-in conversion capabilities. Implementing unit conversions within sdata metadata attributes enables seamless comparison of measurements recorded in heterogeneous units without manual recalculation.

Understanding the sdata Metadata Architecture

The metadata system in sdata consists of two core components defined in sdata/metadata.py. The Attribute class stores individual metadata entries with fields for name, value, unit, and dtype. The Metadata class acts as a container for these attributes, providing methods like set_attr, add, and get to manage the collection.

According to the source code in sdata/metadata.py, attributes already support unit storage via the unit field, but the library only provides extraction helpers like extract_name_unit and set_unit_from_name. To enable actual dimensional transformations—such as converting a force value from kilonewtons to newtons—you must extend the Metadata class with conversion logic.

Adding the convert_attribute Method to Metadata

Lazy Pint Integration Strategy

The implementation relies on Pint (version 0.23 or higher), the de-facto standard for physical quantities in Python. Pint supports NumPy arrays, pandas Series, and scalar values, making it compatible with the numeric data stored in sdata attributes. To keep the core library lightweight, import Pint lazily using a singleton helper function.

Add the following helper to sdata/metadata.py to avoid repeatedly parsing the unit registry:

def _get_ureg():
    """Create (or fetch) a single Pint UnitRegistry."""
    from pint import UnitRegistry
    global _ureg
    try:
        return _ureg
    except NameError:
        _ureg = UnitRegistry()
        return _ureg

The Conversion Implementation

Insert the convert_attribute method into the Metadata class in sdata/metadata.py. This method validates the attribute exists, checks for numeric types, performs the conversion, and mutates the attribute in-place:

from typing import Union

class Metadata(object):
    # ... existing methods ...

    
    def convert_attribute(self, name: str, target_unit: str) -> None:
        """
        Convert the numerical *value* of the attribute ``name`` from its current
        ``unit`` to ``target_unit`` in-place.

        Parameters
        ----------
        name: str
            Attribute identifier (exact match in ``self.attributes``).
        target_unit: str
            Desired unit string understood by Pint, e.g. ``"N"``, ``"kN"``,
            ``"mm"`` or ``"kg·m⁻³"``.

        Raises
        ------
        KeyError
            If ``name`` is not present in the metadata.
        ValueError
            If the attribute has no unit or the conversion is dimensionally
            incompatible.
        """
        attr = self.get(name)
        if attr is None:
            raise KeyError(f"Attribute {name!r} not found in metadata.")
        if not attr.unit or attr.unit == "-":
            raise ValueError(f"Attribute {name!r} has no defined unit.")
        if attr.dtype not in ("float", "int"):
            raise ValueError(
                f"Conversion only supported for numeric types, got {attr.dtype!r}."
            )

        ureg = _get_ureg()
        try:
            q = ureg.Quantity(attr.value, attr.unit)
            q_converted = q.to(target_unit)
        except Exception as exc:
            raise ValueError(
                f"Failed to convert {attr.value} {attr.unit} → {target_unit}: {exc}"
            ) from exc

        # Store the converted value and update the unit string

        attr.value = q_converted.magnitude
        attr.unit = str(q_converted.units)

Remember to add pint>=0.23 to your requirements.txt file to ensure the dependency is available.

Practical Usage Examples

Converting Single Attributes

After implementing the method, convert individual metadata attributes interactively:

>>> from sdata import Data, Metadata
>>> d = Data(name="test", table=None)
>>> d.metadata.add("Force", 1.2, unit="kN", dtype="float", description="applied force")
>>> d.metadata["Force"]
(Attribute 'Force': 1.2(kN))

>>> d.metadata.convert_attribute("Force", "N")
>>> d.metadata["Force"]
(Attribute 'Force': 1200.0(N))

The method updates both the numeric value and the unit string automatically, preserving the original attribute reference.

Bulk Conversion Workflows

For datasets with multiple attributes sharing the same dimension—such as stress components in megapascals that need conversion to pascals—use a simple iteration pattern:

for attr_name in ["Stress_x", "Stress_y", "Stress_z"]:
    d.metadata.convert_attribute(attr_name, "Pa")

This approach ensures consistent units across related measurements without manually recalculating each value.

Integrating with DataFrame Column Parsing

When loading pandas DataFrames, sdata already parses units from column headers using nameunit_from_colname in sdata/data.py. After creating a Data object, normalize all column units to a standard system:

def normalize_units(data_obj, target_units):
    """
    data_obj : sdata.Data instance
    target_units : dict mapping column name → desired unit string
    """
    for col, tgt in target_units.items():
        attr_name = f"_sdata_column_{col}"
        if attr_name in data_obj.metadata:
            data_obj.metadata.convert_attribute(attr_name, tgt)

# Example usage

normalize_units(d, {"Force": "N", "Displacement": "mm"})

This workflow leverages the existing column parsing logic while adding dimensional normalization through the new conversion method.

Testing the Unit Conversion Feature

To guarantee robustness, extend the existing test suite in tests/test_metadata.py with assertions that verify correct magnitude changes and unit string updates:

def test_attribute_conversion():
    md = Metadata()
    md.add("Length", 2.5, unit="cm", dtype="float")
    md.convert_attribute("Length", "m")
    assert md["Length"].value == 0.025
    assert md["Length"].unit == "meter"

def test_conversion_missing_unit_raises():
    md = Metadata()
    md.add("Count", 5, unit="-", dtype="int")
    try:
        md.convert_attribute("Count", "dimensionless")
    except ValueError as e:
        assert "no defined unit" in str(e)

Run these tests with pytest tests/test_metadata.py to ensure the conversion logic handles valid transformations and raises appropriate errors for incompatible dimensions or missing units.

Summary

• Metadata architecture: The Attribute and Metadata classes in sdata/metadata.py already store unit strings but require the convert_attribute method to enable dimensional transformations.
• Pint integration: Implement lazy loading of UnitRegistry via _get_ureg() to keep dependencies optional, adding pint>=0.23 to requirements.txt.
• In-place mutation: The conversion method updates the attribute's value and unit fields directly, maintaining compatibility with existing code that references metadata["name"].value.
• DataFrame workflows: Combine the new method with nameunit_from_colname from sdata/data.py to normalize units across tabular data columns automatically.
• Error handling: Explicit KeyError and ValueError exceptions help detect missing attributes, undefined units, or dimensionally incompatible conversions early.

Frequently Asked Questions

How do I add Pint to my sdata installation?

Add pint>=0.23 to your project's requirements.txt file and install via pip. The convert_attribute method imports Pint lazily through the _get_ureg() helper, so the dependency loads only when you invoke unit conversions.

Can I convert units for attributes stored in DataFrame columns?

Yes. When you create a Data object from a pandas DataFrame, sdata/data.py automatically generates metadata attributes for each column using the nameunit_from_colname helper. Access these attributes using the _sdata_column_{colname} naming convention and call convert_attribute to normalize units before analysis.

What happens if I try to convert between incompatible dimensions?

The convert_attribute method raises a ValueError with a descriptive message when Pint detects dimensionally incompatible units, such as attempting to convert a length attribute to kilograms. This prevents silent data corruption and helps maintain metadata integrity.

Does unit conversion modify the original DataFrame values?

No. The convert_attribute method only modifies the metadata attribute objects stored in the Metadata container. If you need to update the actual data table values, extract the converted magnitude from the metadata attribute and assign it back to your DataFrame column manually.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}