# How AsyncMigrationManager Automatically Runs SurrealDB Schema Migrations on API Startup

> Learn how AsyncMigrationManager automatically runs SurrealDB schema migrations on API startup. Ensure your database is up-to-date before your API handles requests.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-06

---

**When the FastAPI application starts, the `lifespan` context manager in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) instantiates `AsyncMigrationManager`, detects the current schema version from SurrealDB, and executes any pending migrations from `open_notebook/database/migrations/` before the API accepts its first request, aborting startup entirely if any migration fails.**

The `lfnovo/open-notebook` repository guarantees database schema consistency by integrating automatic migration logic into the application boot sequence. Using FastAPI’s asynchronous lifespan hooks, the `AsyncMigrationManager` class orchestrates version detection, migration ordering, and transactional schema updates without requiring manual intervention. This architecture ensures that the SurrealDB schema is always synchronized with the application code before any HTTP endpoints become available.

## The FastAPI Lifespan Hook: Where Automation Begins

In [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py), the `lifespan` async context manager serves as the entry point for schema migration. FastAPI executes this context manager exactly once during startup, before any routes are mounted. Inside this hook, the system initializes the migration manager and triggers the version-check workflow, ensuring the database is ready before the server starts accepting traffic.

```python
from open_notebook.database.async_migrate import AsyncMigrationManager

async def lifespan(app: FastAPI):
    # Database connection setup...

    migration_manager = AsyncMigrationManager()
    current_version = await migration_manager.get_current_version()
    logger.info(f"Current database version: {current_version}")

    if await migration_manager.needs_migration():
        logger.warning("Database migrations are pending. Running migrations...")
        await migration_manager.run_migration_up()
        logger.success(
            f"Migrations completed successfully. Database is now at version "
            f"{await migration_manager.get_current_version()}"
        )
    # Application startup continues...

```

## Initializing the Migration Manager and Scanning Migration Files

The `AsyncMigrationManager` class, defined in [`open_notebook/database/async_migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/async_migrate.py), prepares an ordered list of migrations during instantiation. Its `__init__` method constructs `up_migrations` by explicitly loading each SurrealQL file from `open_notebook/database/migrations/` into an `AsyncMigration` object, ensuring migrations run in sequential order.

```python
class AsyncMigrationManager:
    def __init__(self):
        self.up_migrations = [
            AsyncMigration.from_file("open_notebook/database/migrations/1.surrealql"),
            AsyncMigration.from_file("open_notebook/database/migrations/2.surrealql"),
            # ...

            AsyncMigration.from_file("open_notebook/database/migrations/14.surrealql"),
        ]

```

## Detecting Current Schema Version and Migration Requirements

Before executing any SQL, the manager determines the database’s current state. The `get_current_version` method delegates to `get_latest_version`, which queries the `_sbl_migrations` table in SurrealDB. If this table does not exist, the method returns version `0`. The `needs_migration` method then compares this integer against the length of the `up_migrations` list to determine if the schema is outdated.

## Executing Pending Migrations Transactionally

When migrations are required, `run_migration_up` delegates to the internal runner’s `run_all()` method. This iterates through pending `AsyncMigration` objects, executing each SQL statement via the SurrealDB client using `connection.query(self.sql)`. After each successful migration, `bump_version` updates the schema version in the `_sbl_migrations` table, ensuring atomic progress tracking.

```python
async def run(self, bump: bool = True) -> None:
    async with db_connection() as connection:
        await connection.query(self.sql)  # Execute the SurrealQL

    if bump:
        await bump_version()  # Record new version in _sbl_migrations

    else:
        await lower_version()

```

## Preventing Startup on Migration Failure

If any migration step raises an exception, the lifespan handler in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) catches the error and raises a `RuntimeError`, halting the FastAPI startup process. This fail-fast approach prevents the API from serving requests against an incomplete or incompatible database schema, forcing immediate operational attention to the database state.

## Summary

- **FastAPI lifespan integration**: The `lifespan` context manager in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) triggers `AsyncMigrationManager` exactly once during application startup.
- **Ordered migration scanning**: The manager constructor builds `up_migrations` by loading numbered SurrealQL files from `open_notebook/database/migrations/`.
- **Version detection**: `get_current_version` reads the `_sbl_migrations` table via `get_latest_version` to determine the existing schema state.
- **Migration requirement check**: `needs_migration` compares the current version against the migration file count to identify pending updates.
- **Transactional execution**: `run_migration_up` executes pending statements via `connection.query()` and increments the version using `bump_version`.
- **Startup safety**: Any migration failure raises a `RuntimeError` that aborts startup, preventing the API from running with an outdated schema.

## Frequently Asked Questions

### What triggers the SurrealDB migrations to run when the API starts?

The FastAPI `lifespan` asynchronous context manager defined in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) instantiates `AsyncMigrationManager` and invokes its migration workflow during the application startup sequence. This executes before the server binds to any ports or accepts HTTP requests, ensuring the SurrealDB schema is fully updated and compatible with the running code.

### How does AsyncMigrationManager determine which migrations need to run?

The manager queries the `_sbl_migrations` table via `get_current_version` (which delegates to `get_latest_version`) to retrieve an integer representing the current schema version. The `needs_migration` method compares this value against the length of the internal `up_migrations` list, which contains `AsyncMigration` objects loaded from the filesystem. If the current version is less than the migration count, the system identifies the delta and queues the remaining migrations for execution.

### What happens if a migration fails during API startup?

If any `AsyncMigration` throws an exception during `run_migration_up`, the error propagates to the lifespan handler in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py), which raises a `RuntimeError` and halts the FastAPI startup process. This prevents the application from running with a partially applied schema, ensuring data integrity and forcing immediate resolution of the database issue before the service becomes available.

### Where are the migration SQL files stored in the repository?

Migration files are stored as numbered SurrealQL scripts in the `open_notebook/database/migrations/` directory. The `AsyncMigrationManager` constructor explicitly references files such as `1.surrealql`, `2.surrealql`, and so on, creating an ordered sequence that guarantees schema changes apply in the correct chronological order.