How Metadata Propagation Works Across Nested sdata Structures

Metadata propagation in the lepy/sdata library occurs automatically when you instantiate a Base subclass with parent= or project= arguments, which stores the container's sname in reserved metadata keys (_sdata_parent_sname, _sdata_project_sname) to maintain hierarchical traceability.

The lepy/sdata library manages hierarchical scientific data through a unified Base class that handles lineage tracking without manual intervention. When you nest sdata objects—such as placing a Part inside a Material or tables within a Workbook—the framework propagates critical metadata fields from container to child to ensure data provenance and namespace consistency. This propagation mechanism is implemented entirely within the Base class and its associated Metadata container.

The Core Propagation Mechanism

The propagation logic resides in sdata/base.py and orchestrates how parent and project references flow downstream through object hierarchies via four distinct stages.

Constructor Time Capture

When you instantiate any Base subclass, the __init__ method (lines 31–44) captures the parent= and project= keyword arguments. Rather than storing direct object references, the constructor extracts the sname (human-readable identifier string) from the supplied container and writes it to the instance's metadata under the reserved keys _sdata_parent_sname and _sdata_project_sname.

This approach decouples hierarchical linkage from Python object lifecycles, allowing metadata relationships to persist even when the original parent objects are not loaded in memory.

Reserved Metadata Keys

The Base class defines a class-level list named SDATA_ATTRIBUTES (lines 46–49) that enumerates canonical metadata fields including version, creation time, class type, and the propagation-specific keys. These reserved fields guarantee that every sdata object carries standardized lineage information regardless of its position in the hierarchy or storage state.

Runtime Resolution with SUUID

Retrieving the full parent or project object requires resolving the stored sname back into a complete SUUID. The Base class provides get_parent() and get_project() methods (lines 9–14 and 15–21) that read the cached sname from metadata and invoke SUUID.from_suuid_sname to reconstruct the original identifier. The parent and project properties expose this functionality, returning fully instantiated SUUID objects only when accessed.

Automatic Namespace Inheritance

When you create a child object without explicitly specifying a project argument, the constructor (lines 86–90) automatically falls back to the parent's project context. The code extracts the namespace component from self.project.sname, ensuring that all descendants generate their UUIDs within the same logical project scope and maintain consistent, reproducible identifiers.

Practical Implementation Examples

Creating Hierarchical Object Trees

The following example demonstrates automatic propagation from a project down through materials to individual parts:

from sdata.base import Base

# Create top-level project container

project = Base(name="MyProject")
print(project.sname)  # Output: myproject.0d1e2f...

# Create material within project scope

material = Base(name="Aluminium", project=project)
print(material.metadata.get(Base.SDATA_PROJECT_SNAME).value)  # Matches project.sname

# Create part with material as parent

part = Base(name="Beam", parent=material)
print(part.metadata.get(Base.SDATA_PARENT_SNAME).value)  # Material's sname

print(part.project)  # Resolves to project's SUUID via inheritance

In this sequence, part automatically inherits the project context from material while explicitly recording material as its parent, creating a traceable two-level hierarchy without redundant metadata entry.

Merging Metadata Across Levels

For custom attribute propagation between related objects, use the Metadata container's update methods:

# Add physical property to material

material.metadata.set_attr("density", 2.7, unit="g/cm³", dtype="float")

# Propagate to child component

part.metadata.update_from_dict(material.metadata.to_dict())

print(part.metadata.get("density").value)  # 2.7

The update_from_dict method (implemented in sdata/metadata.py, lines 13–40) preserves reserved hierarchy keys while copying user-defined attributes, preventing accidental corruption of the parent-child relationship chain.

Namespace Consistency Verification

Propagated metadata ensures namespace coherence across the object tree:

print(project.sname.split('.')[0])   # project namespace prefix

print(material.sname.split('.')[0])  # identical prefix

print(part.sname.split('.')[0])      # identical prefix

All objects share the identical namespace prefix because child constructors derive their SUUID namespace from the parent or project sname provided at initialization.

Key Source Files and Functions

Understanding the propagation architecture requires familiarity with these specific implementation locations:

• sdata/base.py: Contains the Base class constructor handling parent= and project= arguments (lines 31–44), the SDATA_ATTRIBUTES definition (lines 46–49), resolution methods get_parent() and get_project() (lines 9–21), and automatic namespace derivation logic (lines 86–90).

• sdata/metadata.py: Implements the Metadata container with update_from_dict() and set_attr() methods (lines 13–40) that facilitate manual metadata synchronization while protecting reserved propagation keys.

• tests/test_base.py: Unit tests validating that _sdata_parent_sname and _sdata_project_sname populate correctly during object instantiation.

• tests/test_ks2.py: Integration tests demonstrating realistic multi-level hierarchies (materials → parts → experiments) and verifying end-to-end propagation behavior.

Summary

• Automatic capture: Supplying parent= or project= to any Base subclass constructor automatically records the container's sname in reserved metadata fields _sdata_parent_sname and _sdata_project_sname.
• Traceable lineage: The parent and project properties resolve stored snames into full SUUID objects, enabling bidirectional hierarchy navigation without direct object references.
• Protected namespace: Child objects inherit project namespaces automatically when no explicit project is provided, ensuring consistent identifier generation across the object tree.
• Safe merging: Use Metadata.update_from_dict() to copy custom attributes between objects without corrupting the reserved hierarchy keys that maintain structural relationships.
• Implementation location: All propagation logic resides in sdata/base.py and sdata/metadata.py within the lepy/sdata repository.

Frequently Asked Questions

How do I manually set a parent relationship after object creation?

Direct modification of _sdata_parent_sname is discouraged because the Base class derives the project namespace during initialization based on the provided parent. Instead, instantiate the object with the parent= argument to ensure consistent propagation. If you must modify an existing object, use metadata.set_attr() with the reserved key constant, then verify that obj.parent resolves correctly via get_parent().

What happens if I provide both parent and project arguments to a constructor?

When both arguments are present, the constructor stores both snames independently in the metadata. The _sdata_parent_sname records the immediate container reference, while _sdata_project_sname records the project scope. If the parent's project differs from the explicitly provided project, the explicit project takes precedence for namespace generation (lines 86–90), though both relationships remain traceable through their respective properties.

Can I prevent automatic project inheritance from a parent object?

Yes. To override inherited project context, explicitly pass a different project= argument when creating the child object. The constructor logic in sdata/base.py only falls back to the parent's project when the argument is omitted, allowing you to place objects in different project scopes while maintaining the parent-child structural relationship via the parent= argument.

How does metadata propagation affect object serialization and storage?

The stored snames in _sdata_parent_sname and _sdata_project_sname serialize as standard string metadata attributes, making hierarchical relationships portable across JSON, HDF5, or other storage formats. When deserializing, these preserved snames enable the get_parent() and get_project() methods to reconstruct the original SUUID relationships, though the actual parent objects must be available in the runtime environment or lookup table for full object resolution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}