# How to Implement Custom Data Types Using sdata's sclass System

> Learn to implement custom data types with sdata's sclass system. Inherit from Base, implement serialization, and register your class for lazy loading and spec resolution.

- Repository: [lepy/sdata](https://github.com/lepy/sdata)
- Tags: how-to-guide
- Published: 2026-03-05

---

**Implementing custom data types in sdata requires inheriting from `sdata.base.Base`, implementing `to_dict`/`from_dict` methods for serialization, and registering the class with `sdata.sclass.register()` to enable lazy loading and canonical spec resolution.**

The `sclass` system in the `lepy/sdata` repository provides a lazy-registry architecture that transforms any Python class into a first-class serializable entity. By implementing custom data types using this system, you gain automatic metadata handling, Super UUID (SUUID) generation, and seamless JSON round-tripping capabilities. This guide walks through the exact steps needed to integrate your custom types into sdata's serialization framework.

## Understanding the sclass Architecture

The sdata `sclass` system centers on a **lazy-registry pattern** that defers imports until runtime, preventing circular dependency issues while maintaining clean JSON references.

### The LazyRegistry Component

At the core of the system is `LazyRegistry`, implemented in [`sdata/sclass/lazy_namespace.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/lazy_namespace.py) (lines 33-75). This class maintains a mapping of short names to fully qualified module specifications (e.g., `"MyThing"` → `"my_pkg.my_mod:MyThing"`). The registry dynamically injects `__getattr__` and `__dir__` into the `sdata.sclass` package, enabling attribute access to trigger on-demand imports.

When you access `sdata.sclass.MyThing`, the registry intercepts the call, splits the stored spec into module and class components, imports the module via `import_module`, and resolves the attribute chain. The resulting class is cached on the package for subsequent accesses, ensuring optimal performance after the initial lazy load.

### Core Registration Functions

The public API for extending the registry lives in [`sdata/sclass/__init__.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/__init__.py) (lines 26-33). The `register(name, spec)` function adds a single entry to the internal `_exports` dictionary, while `register_many` handles batch registrations. These functions modify the global `_registry` instance created during package initialization (lines 7-24).

For serialization workflows, the module provides `class_to_spec` and `spec_to_class` functions (lines 43-61). These utilities convert Python classes to canonical `"module:QualifiedName"` strings and vice versa, enabling sdata to store type information in JSON's `"__spec__"` field.

## Step-by-Step Implementation Guide

Follow these concrete steps to implement a custom data type that integrates fully with sdata's infrastructure.

### 1. Inherit from the Base Class

All sclasses must inherit from `sdata.base.Base` (defined in [`sdata/base.py`](https://github.com/lepy/sdata/blob/main/sdata/base.py), lines 28-35). This base class provides essential infrastructure including metadata dictionaries, SUUID handling, and the `sdata_class` attribute that the registry uses for identification.

```python

# my_pkg/my_mod.py

from sdata.base import Base
import pandas as pd

class MyThing(Base):
    """A custom sclass storing pandas DataFrames with metadata."""
    def __init__(self, df: pd.DataFrame, **kwargs):
        super().__init__(**kwargs)  # Initializes metadata, SUUID, timestamps

        self.data["df"] = df       # Store payload in the data dictionary

```

Inheriting from `Base` automatically attaches metadata fields, generates a unique SUUID, and sets up the object for registry operations.

### 2. Implement Serialization Methods

To support JSON round-tripping via `sdata.sclass.to_json` and `from_json`, implement `to_dict` and `from_dict` methods. These methods handle the conversion of your domain-specific data to serializable primitives.

```python
    @classmethod
    def from_dict(cls, payload: dict):
        """Reconstruct MyThing from deserialized JSON."""
        df = pd.read_json(payload["df"])
        return cls(df=df, name=payload.get("name", "unnamed"))
    
    def to_dict(self):
        """Convert MyThing to a dictionary for JSON serialization."""
        return {
            "df": self.data["df"].to_json(), 
            "name": self.name
        }

```

The `to_dict` method must return JSON-serializable values, while `from_dict` acts as an alternative constructor that reconstructs the object from the deserialized dictionary.

### 3. Register Your Custom sclass

During application startup, register your class with the sdata registry to make it available via lazy import and canonical spec resolution.

```python

# my_pkg/__init__.py or application startup code

from sdata.sclass import register

# Map short name to fully qualified spec

register("MyThing", "my_pkg.my_mod:MyThing")

```

This call adds the entry to `LazyRegistry._exports`, immediately making the class accessible as `sdata.sclass.MyThing` without importing `my_pkg` until the attribute is actually accessed.

### 4. Use in Production Workflows

Once registered, use your custom type through the sclass namespace or serialize it for storage and transmission.

```python
import pandas as pd
from sdata.sclass import MyThing, to_json, from_json

# Instantiate via the lazy-loaded namespace

df = pd.DataFrame({"x": [1, 2, 3], "y": ["a", "b", "c"]})
thing = MyThing(df=df, name="production_data")

# Serialize includes the canonical spec

payload = to_json(thing, include_spec=True)
print(payload["__spec__"])  # Output: my_pkg.my_mod:MyThing

# Deserialize automatically resolves the spec back to MyThing

restored = from_json(payload)
assert isinstance(restored, MyThing)
assert restored.name == "production_data"

```

The `include_spec=True` parameter ensures the JSON contains the `"__spec__"` field, allowing `from_json` to reconstruct the exact class type on deserialization using `spec_to_class`.

## Technical Deep Dive

### Lazy Loading Mechanism

The registry's `_registry.attach()` method (called in [`sdata/sclass/__init__.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/__init__.py)) replaces `sdata.sclass.__getattr__` with a custom implementation. When you reference `sdata.sclass.MyThing`, the custom getter checks `_exports` for the short name, splits the spec string at the colon separator, imports the module portion dynamically, and resolves the attribute chain to retrieve the class.

This design keeps import times minimal because `my_pkg` is not imported when you import `sdata`—only when you actually access `sdata.sclass.MyThing`. The mechanism also prevents circular import issues common in complex data modeling scenarios.

### Canonical Spec Resolution

The canonical spec format (`module.submodule:ClassName`) provides a stable, string-based identifier for classes that persists across process restarts and system boundaries. When `from_json` encounters a payload, it checks for `__spec__` and passes it to `spec_to_class`, which uses `importlib` to load the module and `getattr` chains to resolve nested classes.

If only a short name is present (legacy support), the system falls back to registry resolution via `resolve_by_string` in [`lazy_namespace.py`](https://github.com/lepy/sdata/blob/main/lazy_namespace.py), though modern implementations should always write and read the full canonical spec for unambiguous type identification.

## Summary

- **Inherit from `sdata.base.Base`** to gain metadata, SUUID, and registry compatibility automatically.
- **Implement `to_dict` and `from_dict`** methods to enable JSON serialization of your custom payload data.
- **Register using `sdata.sclass.register()`** with a short name and fully qualified `module:Class` spec to enable lazy loading.
- **Reference files correctly**: Registry logic lives in [`sdata/sclass/__init__.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/__init__.py) and [`sdata/sclass/lazy_namespace.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/lazy_namespace.py), while [`sdata/base.py`](https://github.com/lepy/sdata/blob/main/sdata/base.py) provides the base class.
- **Use canonical specs** (`module:QualifiedName`) for stable serialization and deserialization across systems.

## Frequently Asked Questions

### What is the sclass system in sdata?

The sclass system is a lazy-registry architecture that transforms Python classes into serializable, metadata-rich entities. It enables classes to be referenced by short names, imported on demand, and serialized using canonical specifications. The system lives in the `sdata/sclass` module and relies on `LazyRegistry` for dynamic attribute resolution.

### Why must custom types inherit from Base?

Inheriting from `sdata.base.Base` (defined in [`sdata/base.py`](https://github.com/lepy/sdata/blob/main/sdata/base.py)) provides essential infrastructure including automatic metadata handling, SUUID generation, timestamp tracking, and the `sdata_class` attribute required by the registry. Without this base class, objects lack the standardized interface that `to_json` and `from_json` expect.

### How does the lazy registry prevent circular imports?

The `LazyRegistry` in [`sdata/sclass/lazy_namespace.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/lazy_namespace.py) defers module imports until attribute access occurs. Rather than importing your custom modules when `sdata` loads, the system waits until you explicitly access `sdata.sclass.YourClass`. This breaks import chains because `sdata` never needs to import your application code at startup—your code imports `sdata` instead.

### Can I register multiple classes simultaneously?

Yes. Use the `register_many` function exported from [`sdata/sclass/__init__.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/__init__.py) (lines 30-33) to register multiple name-to-spec mappings in a single call. This function accepts a dictionary mapping short names to canonical specs and is ideal for registering entire modules of custom types during application initialization.