How to Perform Manual Database Migration Execution and Rollback in Open Notebook

To manually execute or rollback database migrations in Open Notebook, instantiate the MigrationManager class from open_notebook.database.migrate and call .up() to apply pending changes or .down() to revert them, optionally passing a target_version parameter to stop at a specific schema version.

Open Notebook persists data in SurrealDB and ships with a versioned migration system that automatically applies schema changes on startup. However, during development, CI/CD pipelines, or disaster recovery, you may need to perform manual database migration execution and rollback to control exactly when and how the schema evolves.

Migration File Structure and Versioning

Migration scripts are stored as SurrealQL files in open_notebook/database/migrations/. Each version consists of an "up" script that applies changes and an optional "down" script that reverts them.

• Up migrations: Named sequentially (1.surrealql, 2.surrealql, … 15.surrealql) containing CREATE, ALTER, or DEFINE statements.
• Down migrations: Named with the _down suffix (1_down.surrealql, …) containing the inverse operations to undo the corresponding up migration.

The migration state is tracked in a SurrealDB table called migration, which stores the current version number and execution history.

Manual Execution via the Synchronous Migration Manager

For Python scripts, maintenance tasks, or interactive shells, use the synchronous wrapper located in open_notebook/database/migrate.py. This class handles the async I/O internally and provides a blocking interface.

Running All Pending Migrations

To bring the database to the latest version, instantiate MigrationManager and call .up():

from open_notebook.database.migrate import MigrationManager

# Creates an AsyncMigrationManager internally and blocks until completion

manager = MigrationManager()

try:
    # Executes all pending up migrations in order (1 → N)

    manager.up()
    print("✅ Database migrated to latest version")
except Exception as exc:
    print(f"❌ Migration failed: {exc}")

Rolling Back the Entire Schema

To revert every applied migration and return to version 0, call .down():

from open_notebook.database.migrate import MigrationManager

manager = MigrationManager()

try:
    # Executes down migrations in reverse order (N → 1)

    manager.down()
    print("✅ Database rolled back to version 0")
except Exception as exc:
    print(f"❌ Rollback failed: {exc}")

Targeting a Specific Version

Both methods accept an optional target_version argument to stop at a specific schema version rather than running all scripts:

from open_notebook.database.migrate import MigrationManager

manager = MigrationManager()

# Migrate up only to version 7, ignoring newer scripts

manager.up(target_version=7)
print("✅ Migrated up to version 7")

# Roll back down to version 3 (executes down scripts for versions 7, 6, 5, 4)

manager.down(target_version=3)
print("✅ Rolled back to version 3")

HTTP API Method for Remote Execution

The migration logic is also exposed via an HTTP endpoint for administrative clients or CI triggers. According to the source code in api/routers/credentials.py (lines 410-435), you can trigger migrations remotely:

# Apply all pending migrations via API

curl -X POST http://localhost:5055/admin/migrate/up

# Roll back all migrations via API

curl -X POST http://localhost:5055/admin/migrate/down

The endpoint utilizes the same MigrationManager internally, ensuring consistency between programmatic and API-driven execution.

Migration Architecture and Safety

The core migration engine resides in open_notebook/database/async_migrate.py, which defines:

• AsyncMigrationManager: Loads migration files from disk and builds a sorted list of pending operations.
• AsyncMigrationRunner: Executes each SurrealQL script within a SurrealDB transaction, ensuring that if any step fails, the entire migration is aborted and the database remains consistent.

If a migration script fails, the system raises a MigrationError and halts execution, leaving the database at the last successfully applied version. You can verify the current state by querying the migration table directly:

SELECT * FROM migration;

Summary

• Manual control: Use MigrationManager from open_notebook/database/migrate.py for synchronous, scriptable migration execution.
• Directional methods: Call .up() to apply pending migrations or .down() to roll them back.
• Version pinning: Pass target_version to migrate or rollback to a specific schema version instead of the extremes.
• Safety: Each migration runs inside a SurrealDB transaction; failures raise MigrationError and prevent partial schema changes.
• Verification: Query the migration table to confirm the current database version after manual operations.

Frequently Asked Questions

Where are the migration scripts stored?

Migration scripts are located in open_notebook/database/migrations/ within the repository. Up-migrations follow the naming convention {version}.surrealql (e.g., 1.surrealql), while rollback scripts use the suffix _down.surrealql (e.g., 1_down.surrealql).

What happens if a migration fails during manual execution?

The AsyncMigrationRunner wraps each migration in a SurrealDB transaction. If any script raises an error, a MigrationError is propagated, the transaction rolls back, and the database remains at the last successfully applied version. The failure is logged, and the migration table is not updated for the failed step.

Can I rollback to a specific version instead of version 0?

Yes. Pass the target_version parameter to manager.down(target_version=3) to roll back migrations in reverse order until the specified version is reached. This allows you to revert specific changes without losing all schema history.

Is the HTTP API endpoint secure for production use?

The migration endpoints are typically mounted under an /admin path within api/routers/credentials.py and should be protected by authentication middleware in production environments. Always verify that your deployment restricts access to these endpoints to authorized administrators only, as executing arbitrary migrations can modify database schema and data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}