# Storage Migration Implications When Updating Deployment Manifests: Akash Network Upgrade Guide

> Learn about storage migration implications when updating Akash Network deployment manifests. Understand manual KV store migration from legacy keys to v2-beta2 IndexedMap format during network upgrades.

- Repository: [Akash Network/node](https://github.com/akash-network/node)
- Tags: migration-guide
- Published: 2026-02-24

---

**Updating deployment manifests in Akash Network requires a state migration during network upgrades because the Cosmos SDK does not automatically rewrite existing KV store entries when protobuf schemas change, necessitating manual migration from `DeploymentV1` legacy keys to the new `v2-beta2` IndexedMap format.**

Akash Network stores deployment specifications (manifests) in the **deployment** module’s KV store, and any structural change—such as moving the **storage** field from the top-level deployment object to `groups[i].spec.storage` in the v2-beta2 specification—triggers a mandatory state migration. Because the Cosmos SDK preserves existing on-chain data layouts, the network must execute a coordinated upgrade that iterates over legacy keys, transforms the data, and writes it to new collection prefixes before new manifest versions can be accepted.

## Why Storage Migrations Are Required for Manifest Updates

The Akash node persists deployment data using protobuf-encoded structs in a key-value database. When the manifest schema evolved from `DeploymentV1` to `DeploymentV2Beta2`, the storage location for volume specifications shifted from the root deployment object into nested group specifications, as documented in [`_docs/adr/adr-002-manifest-v2beta2.md`](https://github.com/akash-network/node/blob/main/_docs/adr/adr-002-manifest-v2beta2.md).

The Cosmos SDK does not perform automatic schema migration on existing state. Consequently, old manifests remain encoded with legacy prefixes (`deployment/…`) and the `DeploymentV1` protobuf representation. Without an explicit migration handler, nodes running the new binary would be unable to locate or decode these legacy entries, leading to state divergence and potential consensus failures.

## The State Migration Execution Flow

When a network upgrade activates a new manifest version, the node executes a deterministic migration sequence that restructures the KV store without data loss.

### Migration Registration and Upgrade Handlers

The upgrade process begins with a governance `Plan` registered in [`app/upgrades.go`](https://github.com/akash-network/node/blob/main/app/upgrades.go), which specifies the target block height and binary version. Each module migration is registered via `utypes.RegisterMigration` located in [`upgrades/types/types.go`](https://github.com/akash-network/node/blob/main/upgrades/types/types.go). For the deployment module’s transition to version 8 (manifest v2-beta2), the registration links the module name to a concrete migration function:

```go
// In upgrades/types/types.go
utypes.RegisterMigration(
    "deployment",           // module name
    8,                     // target version
    func(m utypes.Migrator) utypes.Migration {
        return deploymentMigrations{Migrator: m}
    },
)

```

### KV Store Iterator and Data Transformation

The concrete migration logic resides in [`upgrades/software/v1.2.0/market.go`](https://github.com/akash-network/node/blob/main/upgrades/software/v1.2.0/market.go). The handler constructs a new `collections.IndexedMap` for the updated schema using new prefixes (`mkeys.OrderPrefixNew`, `BidPrefixNew`, `LeasePrefixNew`). It then iterates over all legacy keys using `storetypes.KVStorePrefixIterator` with the old prefixes (`OrderPrefix`, `BidPrefix`, `LeasePrefix`).

For each entry, the handler unmarshals the old protobuf, writes the transformed object into the new collection, and deletes the legacy key. After successful migration, it purges obsolete reverse-lookup keys (`BidPrefixReverse`, `LeasePrefixReverse`). The handler logs migration counts (`orderCount`, `bidCount`, `leaseCount`) via `ctx.Logger().Info("market store migration complete", …)`, providing auditability.

### Atomic Migration and Rollback Safety

The migration operates atomically within the upgrade block. If any step fails—such as a decode error or write failure—the handler returns an error that aborts the entire upgrade, preventing the chain from entering an inconsistent state. Because the old keys remain untouched until successfully written to the new collection, operators can safely re-run the upgrade if a crash occurs mid-migration.

## Critical Risks and Prerequisites

Updating deployment manifests without proper network coordination creates fatal compatibility errors.

### Network Upgrade Coordination

You must schedule and execute the network upgrade **before** submitting any v2-beta2 manifests. The SDK validates incoming transactions against the active protobuf definitions; if the on-chain state still contains the old format, new transactions will be rejected. Submitting a v2-beta2 deployment to a pre-upgrade chain results in immediate transaction failure because the new storage keys do not yet exist.

### Validator Binary Updates

All full nodes and validators must upgrade to the binary version containing the migration handler before the upgrade height. Nodes running legacy software interpret the store using the old schema and will panic when encountering the new `IndexedMap` keys, causing the node to crash and potentially halting consensus if a supermajority of validators is affected.

### Client SDK Compatibility

Client applications must use SDK versions that include the v2-beta2 protobuf definitions. Submitting manifests using outdated client libraries generates transactions encoded with the deprecated `DeploymentV1` layout, which the new chain state will reject post-migration.

## Implementing and Testing Migrations

Developers must pair manifest schema changes with rigorous migration testing.

### Submitting a V2Beta2 Deployment Manifest

When constructing deployments using the Go SDK, explicitly build `DeploymentV2Beta2` objects with storage defined within group specifications:

```go
import (
    akash "github.com/akash-network/go-sdk"
    dtypes "github.com/akash-network/node/x/deployment/types"
)

func submitV2Beta2Deployment(cli *akash.Client, owner sdk.AccAddress) error {
    // Build a DeploymentV2Beta2 protobuf
    dep := &dtypes.DeploymentV2Beta2{
        Groups: []dtypes.Group{
            {
                Spec: &dtypes.GroupSpec{
                    // Storage now lives inside the group spec
                    Volumes: []dtypes.Volume{
                        {
                            Source: dtypes.VolumeSource{
                                Type: dtypes.StorageTypeEphemeral,
                            },
                        },
                    },
                },
            },
        },
    }

    msg := &dtypes.MsgCreateDeployment{
        ID:        dtypes.DeploymentID{Owner: owner.String(), DSeq: 1},
        Deployment: dep,
    }

    _, err := cli.Tx.BroadcastTx(msg)
    return err
}

```

Ensure the client binary is version v3.0.0 or later to include the necessary protobuf definitions.

### Running Local Upgrade Validation

The repository provides an end-to-end test suite to validate migrations before mainnet deployment:

```bash

# From the repository root

make test-upgrade

```

This command executes [`tests/upgrade/upgrade_test.go`](https://github.com/akash-network/node/blob/main/tests/upgrade/upgrade_test.go), which spins up an in-memory node with pre-upgrade deployment data, triggers the upgrade at a simulated height, and verifies that storage keys are correctly rewritten to the new prefixes and that legacy entries are purged.

## Summary

- **State migrations are mandatory** when deployment manifest schemas change because the Cosmos SDK does not rewrite existing KV entries automatically.
- **Migration logic** is implemented in [`upgrades/software/v1.2.0/market.go`](https://github.com/akash-network/node/blob/main/upgrades/software/v1.2.0/market.go) and registered via [`upgrades/types/types.go`](https://github.com/akash-network/node/blob/main/upgrades/types/types.go).
- **Data transformation** uses `KVStorePrefixIterator` to migrate legacy `DeploymentV1` entries into new `collections.IndexedMap` structures with updated prefixes.
- **Upgrade coordination** is critical: all validators must upgrade before the target height, and no v2-beta2 manifests can be submitted until the migration completes successfully.
- **Atomic safety** ensures that failed migrations abort the upgrade, preventing state corruption, while idempotent processing allows restart after crashes.

## Frequently Asked Questions

### What happens if I submit a v2-beta2 manifest before the network upgrade completes?

The transaction will be rejected. The SDK validates manifests against the active on-chain protobuf definitions. Since the new `IndexedMap` keys and `DeploymentV2Beta2` schema do not exist until the migration runs, the transaction fails validation before entering the mempool.

### How does the migration handle crashes or partial failures?

The migration handler records progress counts for each object type (`orderCount`, `bidCount`, `leaseCount`) but leaves original keys untouched until the new entry is successfully written. If the process crashes, operators can restart the node; the upgrade handler will re-execute and skip already-migrated entries. If any irrecoverable error occurs, the handler returns an error that aborts the upgrade, preserving the pre-upgrade state.

### Do all validators need to upgrade simultaneously?

Yes. All validators and full nodes must run the binary containing the migration handler before the upgrade height. Nodes running old software lack the new collection prefixes and will panic when attempting to read state written by upgraded nodes, potentially causing consensus failures and chain halts.

### How can I verify that my deployment data migrated correctly?

Check the node logs for the migration completion message: `ctx.Logger().Info("market store migration complete", …)`. Additionally, run the built-in upgrade tests using `make test-upgrade`, which validates that legacy keys are removed and new `IndexedMap` entries contain correctly transformed data.