Dolt Schema Evolution and Migration: How Version-Controlled Schemas Work

Dolt detects schema changes by comparing deterministic schema hashes and uses a three-way merge strategy to reconcile divergent schema edits across branches, surfacing conflicts through a structured SchemaConflict object when automatic resolution fails.

Dolt schema evolution and migration capabilities allow teams to branch, merge, and track database schema changes with the same version control semantics used for application code. In the dolthub/dolt repository, this functionality is implemented through hash-based change detection in go/libraries/doltcore/diff and sophisticated three-way merge algorithms in go/libraries/doltcore/merge that preserve data integrity across schema transformations.

Detecting Schema Changes with TableDelta

Dolt identifies schema modifications by comparing deterministic hashes of the schema definition. When building a list of table deltas, the system invokes TableDelta.HasSchemaChanged in go/libraries/doltcore/diff/table_deltas.go to check for any structural alterations:

func (td TableDelta) HasSchemaChanged(ctx context.Context) (bool, error) {
    // … guards against non-table objects, checks for adds/drops, FK changes …
    fromSchemaHash, err := td.FromTable.GetSchemaHash(ctx)
    toSchemaHash,   err := td.ToTable.GetSchemaHash(ctx)
    return !fromSchemaHash.Equal(toSchemaHash), nil
}

Source: table_deltas.go:54‑98

This method first validates that the objects are tables, then checks for additions, drops, foreign-key changes, and auto-increment alterations. The canonical check compares fromSchemaHash against toSchemaHash. If the hashes differ, Dolt flags the table for migration and triggers the merge resolution pipeline.

Migration Helpers for Backward Compatibility

When Dolt's internal representation of table identifiers evolved from plain strings to the richer doltdb.TableName struct (which includes schema names), the codebase required migration helpers to maintain backward compatibility with existing repositories. These utilities live in go/libraries/doltcore/doltdb/root_val.go:

// ToTableNames converts a slice of raw names into TableName structs.
func ToTableNames(names []string, schemaName string) []TableName {
    tbls := make([]TableName, len(names))
    for i, name := range names {
        tbls[i] = TableName{Name: name, Schema: schemaName}
    }
    return tbls
}

Source: root_val.go:24‑31

The complementary FlattenTableNames function strips schema information when only plain names are required. These helpers ensure that older persisted data—such as table name lists stored in commit metadata—migrates safely to new struct-based representations. For example, when staging tables via dolt add, the system converts raw names using:

roots, err = actions.StageTables(ctx, roots,
    doltdb.ToTableNames(meta.TablesToStage, doltdb.DefaultSchemaName), false)

Source: dolt_add.go:125‑128

Three-Way Schema Merge Resolution

When two branches modify the same table's schema, Dolt performs a three-way merge to reconcile the divergent definitions. The entry point is SchemaMerge in go/libraries/doltcore/merge/merge_schema.go:

func SchemaMerge(
    ctx *sql.Context,
    format *storetypes.NomsBinFormat,
    ourSch, theirSch, ancSch schema.Schema,
    tblName doltdb.TableName,
) (sch schema.Schema, sc SchemaConflict, mergeInfo MergeInfo,
   diffInfo tree.ThreeWayDiffInfo, err error) {
    // … core logic …
}

Source: merge_schema.go:65‑78

The merge process follows a strict protocol:

1. Fast-path optimization – If ourSch, theirSch, and ancSch are identical, the function returns immediately without processing.
2. Primary-key validation – Dolt currently cannot merge tables whose primary-key sets differ, aborting early with ErrMergeWithDifferentPks.
3. Column-level merging – The mergeColumns function reconciles column additions, deletions, and type changes, returning a merged collection plus any column-level conflicts.
4. Index and constraint resolution – Index modifications and check-constraint changes are merged separately, potentially producing IdxConflict or ChkConflict objects.

All detected conflicts aggregate into a SchemaConflict struct:

type SchemaConflict struct {
    TableName            doltdb.TableName
    ColConflicts         []ColConflict
    IdxConflicts         []IdxConflict
    ChkConflicts         []ChkConflict
    ModifyDeleteConflict bool
}

Source: merge_schema.go:51‑58

When conflicts exist, the merge aborts with a human-readable error:

merge aborted: schema conflict found for table mytable 
 please resolve schema conflicts before merging: …

Source: merge_schema.go:78‑86

Migration Logic for Stored Procedures

Dolt extends its schema migration pattern to version-controlled SQL stored procedures. When procedures are added, removed, or altered in the _procedures system table, the procedures_table implementation triggers the same migration path used for table schemas. The test suite in go/libraries/doltcore/sqle/procedures_table_test.go validates this behavior:

t.Run("test migration logic", func(t *testing.T) { … })
t.Run("test that fetching stored procedure triggers the migration logic", func(t *testing.T) { … })
t.Run("test that adding a new stored procedure triggers the migration logic", func(t *testing.T) { … })

Source: procedures_table_test.go:43‑80

This ensures that changes to stored procedures automatically migrate to the current underlying storage format, maintaining backward compatibility across Dolt versions.

Programmatic Schema Migration Example

You can invoke Dolt's schema merge logic programmatically to build custom migration tools. The following example demonstrates loading three schema versions and executing a three-way merge:

import (
    "context"
    "fmt"
    "github.com/dolthub/dolt/go/libraries/doltcore/merge"
    "github.com/dolthub/dolt/go/libraries/doltcore/schema"
    "github.com/dolthub/dolt/go/libraries/doltcore/doltdb"
    "github.com/dolthub/go-mysql-server/sql"
)

// mergeTableSchema reconciles schema changes between branches.
func mergeTableSchema(
    ctx context.Context,
    ourRoot, theirRoot, ancRoot doltdb.RootValue,
    tblName doltdb.TableName,
) error {
    // Load the three schema objects from their respective roots.
    ourSch,   _ := ourRoot.GetTableSchema(ctx, tblName)
    theirSch, _ := theirRoot.GetTableSchema(ctx, tblName)
    ancSch,   _ := ancRoot.GetTableSchema(ctx, tblName)

    // Execute the three-way schema merge.
    merged, conflict, _, _, err := merge.SchemaMerge(
        sql.NewContext(ctx),
        ourRoot.Format(),
        ourSch, theirSch, ancSch,
        tblName,
    )
    if err != nil {
        return err
    }
    if conflict.Count() > 0 {
        return fmt.Errorf("schema conflict: %s", conflict)
    }

    // Persist the merged schema back to a new root value.
    newRoot, err := ourRoot.PutTableSchema(ctx, tblName, merged)
    if err != nil {
        return err
    }
    _ = newRoot // Commit newRoot via doltdb.Commit to finalize
    return nil
}

This pattern leverages merge.SchemaMerge, TableDelta.HasSchemaChanged, and root_val.go write operations to integrate Dolt's versioned schema capabilities into external applications.

Summary

• Schema hashes serve as the canonical source of truth for detecting changes; any modification to columns, types, defaults, or indexes produces a new hash detectable by TableDelta.HasSchemaChanged.
• Migration helpers (ToTableNames, FlattenTableNames) in go/libraries/doltcore/doltdb/root_val.go ensure backward compatibility when internal representations evolve.
• Three-way schema merging via SchemaMerge automatically reconciles non-conflicting edits across branches while surfacing genuine conflicts through the SchemaConflict struct.
• Stored procedure changes trigger the same migration patterns as table schemas, ensuring consistent version control across all database objects.

Frequently Asked Questions

How does Dolt detect schema changes between commits?

Dolt detects schema changes by comparing deterministic schema hashes stored in the table metadata. The TableDelta.HasSchemaChanged method in go/libraries/doltcore/diff/table_deltas.go retrieves the schema hash from both the source and destination tables using GetSchemaHash, then returns true if the hashes differ. This approach captures any alteration to columns, indexes, constraints, or defaults.

What happens when two branches modify the same table schema?

When branches diverge on schema definitions, Dolt executes a three-way merge through SchemaMerge in go/libraries/doltcore/merge/merge_schema.go. The function compares the common ancestor schema against both branch versions, attempting to auto-merge column additions, type changes, and index modifications. If the changes conflict—for example, both branches add a column with the same name but different types—Dolt returns a SchemaConflict object and aborts the merge with a descriptive error message.

Can Dolt merge tables with different primary keys?

No. Dolt currently cannot perform schema merges when the primary-key sets differ between branches. The SchemaMerge function validates primary-key compatibility early in the process and aborts with ErrMergeWithDifferentPks if the sets do not match exactly. This restriction prevents data loss during the merge process.

How does Dolt handle migration of system tables like stored procedures?

Dolt applies the same migration framework used for user tables to system tables storing stored procedures. When the _procedures table changes—whether through addition, deletion, or modification—the system triggers migration logic in go/libraries/doltcore/sqle/procedures_table.go. This ensures that procedure definitions automatically upgrade to the current storage format while maintaining backward compatibility with older repository versions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}