# What Is AsyncMigrationManager? Database Schema Management in Open Notebook

> Discover AsyncMigrationManager, the core component automating SurrealDB schema migrations. Learn how it collects SQL scripts and executes pending updates on app startup.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: internals
- Published: 2026-06-11

---

**AsyncMigrationManager is the central async coordinator that automates SurrealDB schema migrations by collecting SQL scripts, detecting current versions, and executing pending updates on application startup.**

In the Open Notebook project, database schema evolution is handled automatically through the `AsyncMigrationManager` class. This component ensures that the SurrealDB backend stays synchronized with application code by running migrations safely in an async context whenever the service starts.

## Core Responsibilities of AsyncMigrationManager

The `AsyncMigrationManager` class in [`open_notebook/database/async_migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/async_migrate.py) orchestrates six critical operations for schema management:

### Collecting Migration Scripts

When initialized, the manager scans the `open_notebook/database/migrations/` directory to build ordered lists of migration objects. It separates **up** migrations (forward changes) from **down** migrations (rollback scripts), storing them in `self.up_migrations` and `self.down_migrations` respectively.

### Detecting Current Schema Version

The manager queries the `_sbl_migrations` table using the `get_latest_version()` method to determine the most recently applied migration. This version check happens in [`open_notebook/database/async_migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/async_migrate.py) lines 98-105, where it executes a SELECT against the bookkeeping table.

### Determining Migration Requirements

The `needs_migration()` method compares the current database version against the number of available **up** migrations. If the counts differ, the system knows pending changes exist and returns `True` to trigger execution.

### Executing Pending Migrations

When migrations are needed, `run_migration_up()` invokes the `AsyncMigrationRunner` to apply each missing script sequentially. This process includes error handling and progress logging to ensure atomic application of schema changes.

### Version Bookkeeping

After each successful migration, the `bump_version()` helper inserts a new row into `_sbl_migrations` (lines 20-28 in [`async_migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/async_migrate.py)). This creates an immutable audit trail of all schema changes applied to the database.

### Automatic Startup Integration

The FastAPI lifespan hook in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) (lines 17-33) instantiates `AsyncMigrationManager`, checks `needs_migration()`, and automatically runs `run_migration_up()` before the API begins serving requests.

## How AsyncMigrationManager Integrates with FastAPI

Open Notebook ensures zero-downtime deployments by running migrations during the application lifespan. The integration in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) follows this pattern:

```python
from open_notebook.database.async_migrate import AsyncMigrationManager

async def lifespan(app):
    manager = AsyncMigrationManager()
    if await manager.needs_migration():
        await manager.run_migration_up()
    # ... continue with startup

```

This guarantees that the database schema is always up-to-date before handling traffic, eliminating manual migration steps during deployment.

## Manual Migration Operations

While automatic execution handles most cases, you can interact with `AsyncMigrationManager` directly for maintenance tasks.

### Checking Migration Status

To verify current version and pending changes:

```python
async def check_status():
    manager = AsyncMigrationManager()
    current = await manager.get_current_version()
    pending = len(manager.up_migrations) - current
    print(f"Current version: {current}, Pending: {pending}")

```

### Rolling Back Migrations

To reverse the last applied migration using the down scripts:

```python
async def rollback():
    manager = AsyncMigrationManager()
    await manager.runner.run_one_down()

```

### Adding New Migrations

When extending the schema:

1. Create a new SQL file in `open_notebook/database/migrations/` (e.g., `015_add_new_table.surrealql`)
2. Add the file path to `self.up_migrations` in `AsyncMigrationManager.__init__`
3. Include a corresponding down script in `self.down_migrations` for rollback capability

## Synchronous Compatibility Layer

For legacy synchronous code, the repository provides a wrapper in [`open_notebook/database/migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/migrate.py). The `Migration` class proxies calls to `AsyncMigrationManager`:

```python
from open_notebook.database.migrate import Migration

# Synchronous API that delegates to the async manager

Migration().run()

```

## Summary

- **AsyncMigrationManager** coordinates all SurrealDB schema migrations in Open Notebook through [`open_notebook/database/async_migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/async_migrate.py)
- It maintains version state in the `_sbl_migrations` table and uses `get_latest_version()` to detect the current schema level
- The `needs_migration()` method determines if pending updates exist by comparing current version against available scripts
- **Up migrations** apply forward changes via `run_migration_up()`, while **down migrations** handle rollbacks through `AsyncMigrationRunner`
- FastAPI lifespan hooks automatically execute migrations on startup, ensuring schema consistency before serving requests
- Version tracking occurs through `bump_version()`, which records each successful migration in the database

## Frequently Asked Questions

### How does AsyncMigrationManager know which migrations to run?

The manager scans the `open_notebook/database/migrations/` directory during initialization to build ordered lists of available scripts. It compares the highest available **up** migration index against the current version stored in the `_sbl_migrations` table. Any scripts with indices higher than the current version are queued for execution via `run_migration_up()`.

### What happens if a migration fails during automatic startup?

If `AsyncMigrationRunner` encounters an error while executing a migration script, the exception propagates up through `run_migration_up()` and halts the FastAPI startup process. This prevents the application from serving requests with an incomplete or corrupted schema, ensuring atomic migration application.

### Can I run migrations manually without starting the full application?

Yes. You can instantiate `AsyncMigrationManager` directly in a Python script or REPL and call `await manager.run_migration_up()` to execute pending migrations. For synchronous environments, use the `Migration` class from [`open_notebook/database/migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/migrate.py) which provides a blocking wrapper around the async manager.

### Where are migration versions stored and tracked?

Version history persists in the `_sbl_migrations` table within SurrealDB. The `get_latest_version()` method queries this table to determine the current schema state, while `bump_version()` inserts new records after each successful migration. This table serves as the single source of truth for schema versioning across all deployment environments.