How the LoopX State Migration Module Handles Goal State Schema Evolution

The LoopX state_migration module migrates legacy goal state documents by inspecting schema_version tags, applying deterministic transformation packets, and enforcing validation constraints before persisting upgraded versions back to the event store.

The state_migration module in the huangruiteng/loopx repository provides the infrastructure for evolving goal state schemas without breaking existing deployments. When the LoopX control plane persists goal states or todo sources to the event store, it embeds a schema_version identifier that enables the runtime to detect legacy documents and apply targeted transformations. This approach ensures that workers, extensions, and coordination logic always interact with the latest schema while preserving historical data.

Schema Version Tagging and Migration Packets

Every persisted goal state document and sub-object—such as a Todo source—carries a schema_version field that acts as a compatibility contract. When developers introduce breaking changes, they define a new version identifier (e.g., "todo_source_section_migration_v0") and register a corresponding migration packet.

In tests/test_goal_terminal_no_followup.py, the test suite validates that legacy todo sources expose the expected version tag:

# Legacy document structure detected in test assertions (lines 143-146)

{
    "todo_source_migration": {
        "schema_version": "todo_source_section_migration_v0",
        # ... additional legacy fields

    }
}

The migration packet itself is a dictionary that maps the legacy schema_version to transformation rules, replacement identifiers, and behavioral flags required to upgrade the document to the current schema.

Migration Packet Structure and Validation Constraints

Migration packets in LoopX are structured data objects that specify exactly how to transform a legacy document. According to the test specifications in tests/extensions/test_finance_value_discovery_extension.py (lines 702-732), each packet contains:

• schema_version: The target version identifier after migration completes.
• replacement_extension_id: The string ID of the modern extension that supersedes the legacy implementation (e.g., "loopx-finance-value-discovery").
• automatic_migration: A boolean flag indicating whether the upgrade can proceed without operator intervention.
• automatic_provider_install_supported: A flag denoting if the migration can automatically install required providers.

When the runtime encounters a document with an outdated schema_version, it retrieves the corresponding packet and validates that all constraints are satisfied. If automatic_migration is set to false, the module raises a descriptive error that halts processing and instructs the operator to perform manual steps before retrying.

Automatic Versus Manual Migration Paths

The state_migration module distinguishes between automatic migrations—which rewrite documents deterministically in-place—and manual migrations that require external validation or provider installation. This distinction prevents accidental data corruption when semantic changes require human review. The runtime checks these flags through pure validation functions before invoking any transformation logic.

Core Migration Functions and Runtime Integration

The module exposes pure functions that perform the actual document transformations. These functions are deterministic and side-effect free, making them safe to invoke from workers, CLI tools, or HTTP handlers.

migrate_goal_state(old_state): Accepts a legacy goal state dictionary, validates its migration packet, and returns a new dictionary conforming to the current schema.

migrate_todo_source(legacy_doc): Specialized handler for todo source objects that manages field renames, default value injection, and extension ID swaps.

from loopx.control_plane.runtime.state_migration import migrate_todo_source

legacy_doc = {
    "id": "todo-1",
    "schema_version": "todo_source_section_migration_v0",
    "tasks": [],
}

# Validates packet constraints and returns upgraded document

upgraded_doc = migrate_todo_source(legacy_doc)
print(upgraded_doc["schema_version"])  # Output: todo_source_section_migration_v1

Event Store Integration

When the control plane loads a goal state from the event store, the loader automatically checks the embedded schema_version. If the version predates the current schema, the loader invokes the migration bridge before returning the object to application code. This pattern, verified in tests/control_plane/test_event_store_migration_bridge.py, guarantees that business logic never processes stale schemas.

Idempotent State Upgrades

Migration operations are designed to be idempotent: applying the same migration packet to an already-upgraded document is a no-op. This safety property ensures that retries after partial failures do not corrupt state. The test suite in tests/control_plane/test_coordination_recoverable_execution.py enforces this behavior by verifying that legacy heads cannot be claimed until migration completes successfully.

Practical Implementation Examples

The following patterns demonstrate how developers interact with the migration system when building extensions or debugging state issues.

Migrating a Full Goal State Automatically:

from loopx.control_plane.runtime.state_migration import migrate_goal_state

raw_state = load_from_event_store(goal_id)  # Contains legacy schema_version

state_v2 = migrate_goal_state(raw_state)    # Applies all needed packets

persist(state_v2)                          # Stores upgraded version

Inspecting Migration Packet Metadata:

from loopx.extensions.migration_registry import get_migration_packet

packet = get_migration_packet(
    legacy_schema="value_connector_extension_migration_v0"
)

print(packet["replacement_extension_id"])  # loopx-finance-value-discovery

print(packet["automatic_migration"])       # False (requires manual action)

Handling Manual Migration Requirements:

from loopx.control_plane.runtime.state_migration import MigrationRequired

try:
    upgraded = migrate_goal_state(legacy_payload)
except MigrationRequired as e:
    # Log operator instructions from e.message

    trigger_alert(f"Manual migration required for goal {goal_id}")

Summary

• The state_migration module uses version tagging (schema_version) to identify legacy goal state documents stored in the LoopX event store.
• Migration packets define transformation rules, replacement extension IDs, and automation flags that govern how each legacy version upgrades to the current schema.
• The module provides pure migration functions (migrate_goal_state, migrate_todo_source) that perform deterministic, idempotent document transformations.
• Automatic and manual migration paths are distinguished by boolean flags in the migration packet, ensuring unsafe upgrades require explicit operator approval.
• Event store integration automatically triggers migration when loading historical documents, guaranteeing that runtime code always receives schema-compliant data.

Frequently Asked Questions

What triggers a goal state migration in LoopX?

A migration triggers when the control plane loads a document from the event store and detects a schema_version that does not match the current runtime schema. The loader consults the migration registry, retrieves the appropriate packet for that legacy version, and either applies the transformation automatically or raises a MigrationRequired exception if manual intervention is necessary.

How does the state_migration module ensure data integrity during upgrades?

The module enforces data integrity through idempotent transformations and constraint validation. Each migration packet specifies required conditions (such as automatic_migration: true) that must be met before any writes occur. Because the core functions are pure and deterministic, rerunning a migration on an already-migrated document returns the same result without side effects, preventing corruption during retries.

Can migrations be performed manually by operators?

Yes. When a migration packet has automatic_migration set to false, the runtime halts and emits a detailed error message instructing the operator to perform prerequisite steps—such as installing specific providers or validating configuration—before attempting the upgrade again. This design prevents accidental automatic upgrades that could break business logic.

Where is the migration logic defined for specific schema versions?

Migration logic is defined in the state_migration module under loopx/control_plane/runtime/state_migration, with concrete packet definitions and test assertions located in files such as tests/test_goal_terminal_no_followup.py and tests/extensions/test_finance_value_discovery_extension.py. These files map legacy schema_version strings (like "todo_source_section_migration_v0") to their corresponding transformation rules and replacement extension IDs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}