# How the LoopX State Migration Module Handles Goal State Schema Evolution

> Discover how LoopX state migration ensures schema evolution for goal states. Learn about version inspection, transformation packets, and validation for event store upgrades.

- Repository: [huangruiteng/loopx](https://github.com/huangruiteng/loopx)
- Tags: internals
- Published: 2026-09-02

---

**The LoopX `state_migration` module migrates legacy goal state documents by inspecting `schema_version` tags, applying deterministic transformation packets, and enforcing validation constraints before persisting upgraded versions back to the event store.**

The `state_migration` module in the [huangruiteng/loopx](https://github.com/huangruiteng/loopx) repository provides the infrastructure for evolving goal state schemas without breaking existing deployments. When the LoopX control plane persists goal states or todo sources to the event store, it embeds a `schema_version` identifier that enables the runtime to detect legacy documents and apply targeted transformations. This approach ensures that workers, extensions, and coordination logic always interact with the latest schema while preserving historical data.

## Schema Version Tagging and Migration Packets

Every persisted goal state document and sub-object—such as a **Todo source**—carries a `schema_version` field that acts as a compatibility contract. When developers introduce breaking changes, they define a new version identifier (e.g., `"todo_source_section_migration_v0"`) and register a corresponding migration packet.

In [`tests/test_goal_terminal_no_followup.py`](https://github.com/huangruiteng/loopx/blob/main/tests/test_goal_terminal_no_followup.py), the test suite validates that legacy todo sources expose the expected version tag:

```python

# Legacy document structure detected in test assertions (lines 143-146)

{
    "todo_source_migration": {
        "schema_version": "todo_source_section_migration_v0",
        # ... additional legacy fields

    }
}

```

The migration packet itself is a dictionary that maps the legacy `schema_version` to transformation rules, replacement identifiers, and behavioral flags required to upgrade the document to the current schema.

## Migration Packet Structure and Validation Constraints

Migration packets in LoopX are structured data objects that specify exactly how to transform a legacy document. According to the test specifications in [`tests/extensions/test_finance_value_discovery_extension.py`](https://github.com/huangruiteng/loopx/blob/main/tests/extensions/test_finance_value_discovery_extension.py) (lines 702-732), each packet contains:

- **`schema_version`**: The target version identifier after migration completes.
- **`replacement_extension_id`**: The string ID of the modern extension that supersedes the legacy implementation (e.g., `"loopx-finance-value-discovery"`).
- **`automatic_migration`**: A boolean flag indicating whether the upgrade can proceed without operator intervention.
- **`automatic_provider_install_supported`**: A flag denoting if the migration can automatically install required providers.

When the runtime encounters a document with an outdated `schema_version`, it retrieves the corresponding packet and validates that all constraints are satisfied. If `automatic_migration` is set to `false`, the module raises a descriptive error that halts processing and instructs the operator to perform manual steps before retrying.

### Automatic Versus Manual Migration Paths

The `state_migration` module distinguishes between **automatic migrations**—which rewrite documents deterministically in-place—and **manual migrations** that require external validation or provider installation. This distinction prevents accidental data corruption when semantic changes require human review. The runtime checks these flags through pure validation functions before invoking any transformation logic.

## Core Migration Functions and Runtime Integration

The module exposes pure functions that perform the actual document transformations. These functions are deterministic and side-effect free, making them safe to invoke from workers, CLI tools, or HTTP handlers.

**`migrate_goal_state(old_state)`**: Accepts a legacy goal state dictionary, validates its migration packet, and returns a new dictionary conforming to the current schema.

**`migrate_todo_source(legacy_doc)`**: Specialized handler for todo source objects that manages field renames, default value injection, and extension ID swaps.

```python
from loopx.control_plane.runtime.state_migration import migrate_todo_source

legacy_doc = {
    "id": "todo-1",
    "schema_version": "todo_source_section_migration_v0",
    "tasks": [],
}

# Validates packet constraints and returns upgraded document

upgraded_doc = migrate_todo_source(legacy_doc)
print(upgraded_doc["schema_version"])  # Output: todo_source_section_migration_v1

```

### Event Store Integration

When the control plane loads a goal state from the event store, the loader automatically checks the embedded `schema_version`. If the version predates the current schema, the loader invokes the migration bridge before returning the object to application code. This pattern, verified in [`tests/control_plane/test_event_store_migration_bridge.py`](https://github.com/huangruiteng/loopx/blob/main/tests/control_plane/test_event_store_migration_bridge.py), guarantees that business logic never processes stale schemas.

### Idempotent State Upgrades

Migration operations are designed to be **idempotent**: applying the same migration packet to an already-upgraded document is a no-op. This safety property ensures that retries after partial failures do not corrupt state. The test suite in [`tests/control_plane/test_coordination_recoverable_execution.py`](https://github.com/huangruiteng/loopx/blob/main/tests/control_plane/test_coordination_recoverable_execution.py) enforces this behavior by verifying that legacy heads cannot be claimed until migration completes successfully.

## Practical Implementation Examples

The following patterns demonstrate how developers interact with the migration system when building extensions or debugging state issues.

**Migrating a Full Goal State Automatically:**

```python
from loopx.control_plane.runtime.state_migration import migrate_goal_state

raw_state = load_from_event_store(goal_id)  # Contains legacy schema_version

state_v2 = migrate_goal_state(raw_state)    # Applies all needed packets

persist(state_v2)                          # Stores upgraded version

```

**Inspecting Migration Packet Metadata:**

```python
from loopx.extensions.migration_registry import get_migration_packet

packet = get_migration_packet(
    legacy_schema="value_connector_extension_migration_v0"
)

print(packet["replacement_extension_id"])  # loopx-finance-value-discovery

print(packet["automatic_migration"])       # False (requires manual action)

```

**Handling Manual Migration Requirements:**

```python
from loopx.control_plane.runtime.state_migration import MigrationRequired

try:
    upgraded = migrate_goal_state(legacy_payload)
except MigrationRequired as e:
    # Log operator instructions from e.message

    trigger_alert(f"Manual migration required for goal {goal_id}")

```

## Summary

- The `state_migration` module uses **version tagging** (`schema_version`) to identify legacy goal state documents stored in the LoopX event store.
- **Migration packets** define transformation rules, replacement extension IDs, and automation flags that govern how each legacy version upgrades to the current schema.
- The module provides **pure migration functions** (`migrate_goal_state`, `migrate_todo_source`) that perform deterministic, idempotent document transformations.
- **Automatic and manual migration paths** are distinguished by boolean flags in the migration packet, ensuring unsafe upgrades require explicit operator approval.
- **Event store integration** automatically triggers migration when loading historical documents, guaranteeing that runtime code always receives schema-compliant data.

## Frequently Asked Questions

### What triggers a goal state migration in LoopX?

A migration triggers when the control plane loads a document from the event store and detects a `schema_version` that does not match the current runtime schema. The loader consults the migration registry, retrieves the appropriate packet for that legacy version, and either applies the transformation automatically or raises a `MigrationRequired` exception if manual intervention is necessary.

### How does the state_migration module ensure data integrity during upgrades?

The module enforces data integrity through **idempotent transformations** and **constraint validation**. Each migration packet specifies required conditions (such as `automatic_migration: true`) that must be met before any writes occur. Because the core functions are pure and deterministic, rerunning a migration on an already-migrated document returns the same result without side effects, preventing corruption during retries.

### Can migrations be performed manually by operators?

Yes. When a migration packet has `automatic_migration` set to `false`, the runtime halts and emits a detailed error message instructing the operator to perform prerequisite steps—such as installing specific providers or validating configuration—before attempting the upgrade again. This design prevents accidental automatic upgrades that could break business logic.

### Where is the migration logic defined for specific schema versions?

Migration logic is defined in the `state_migration` module under `loopx/control_plane/runtime/state_migration`, with concrete packet definitions and test assertions located in files such as [`tests/test_goal_terminal_no_followup.py`](https://github.com/huangruiteng/loopx/blob/main/tests/test_goal_terminal_no_followup.py) and [`tests/extensions/test_finance_value_discovery_extension.py`](https://github.com/huangruiteng/loopx/blob/main/tests/extensions/test_finance_value_discovery_extension.py). These files map legacy `schema_version` strings (like `"todo_source_section_migration_v0"`) to their corresponding transformation rules and replacement extension IDs.