How Podcast Profile Migration Converts Legacy String Profiles to Model Registry Records

The podcast profile migration automatically runs on API startup to convert legacy string-based provider and model configurations into structured Model registry records.

Open Notebook handles this transition seamlessly in the lfnovo/open-notebook repository, ensuring that existing EpisodeProfile and SpeakerProfile records migrate from free-form text fields to proper SurrealDB record references. This podcast profile migration bridges the gap between the old configuration format and the newer credential-driven Model registry system.

How the Migration Works

The migration logic lives in open_notebook/podcasts/migration.py and executes after SQL schema migrations complete. It processes both podcast profile tables to identify legacy configurations and upgrade them to reference concrete Model records.

Entry Point and Bootstrap

The migrate_podcast_profiles() function serves as the entry point, invoked during application startup in api/main.py. When the API initializes, this function logs the migration start and then sequentially processes the episode_profile and speaker_profile tables.

Finding or Creating Model Records

At the heart of the migration, _find_model_record() queries the model table for an exact match on provider, name, and type parameters. If no existing record is found, _find_or_create_model() takes over:

• It calls Credential.get_by_provider() to locate stored credentials for the specified provider
• Upon finding a valid credential, it instantiates a new Model object using the class defined in open_notebook/ai/models.py
• The new model persists via await model.save() and returns its ID
• The migration wraps this ID with ensure_record_id (from open_notebook/database/repository.py) to guarantee a proper SurrealDB record reference format

EpisodeProfile Migration

For each row in the episode_profile table, the code inspects the outline_llm and transcript_llm fields. When either field is missing or empty, the migration reads the legacy *_provider and *_model string values, converts them to a Model ID using _find_or_create_model(), and updates the record via repo_update("episode_profile", …, updates).

SpeakerProfile Migration

The SpeakerProfile migration follows an identical pattern. If the voice_model field is empty, the code extracts the legacy tts_provider and tts_model strings, obtains a corresponding Model ID, and persists the update back to the database.

Idempotence and Safety

The migration skips any row that already contains populated *_llm or voice_model fields, making repeated runs safe and non-destructive. All operations—successes, skips, and failures—are logged for complete auditability.

Code Implementation Details

The migration relies on specific components across the codebase:

• open_notebook/podcasts/migration.py – Contains _find_model_record(), _find_or_create_model(), and the main migrate_podcast_profiles() orchestration logic
• open_notebook/ai/models.py – Defines the Model class used for registry entries
• open_notebook/domain/credential.py – Provides Credential.get_by_provider() for linking providers to API keys
• open_notebook/database/repository.py – Supplies low-level helpers including repo_query, repo_update, and ensure_record_id
• api/main.py – Triggers the migration during API startup

Practical Examples

Trigger the migration manually (normally executed automatically):

from open_notebook.podcasts.migration import migrate_podcast_profiles

await migrate_podcast_profiles()

After migration, an EpisodeProfile contains structured record references:

{
    "id": "episode_profile:123",
    "outline_llm": "model:45",          # SurrealDB record reference

    "transcript_llm": "model:46",
    ...
}

Creating a Model manually (as the migration does when no match exists):

from open_notebook.ai.models import Model
from open_notebook.domain.credential import Credential

cred = await Credential.get_by_provider("openai")  # pick first credential

model = Model(
    name="gpt-4o-mini",
    provider="openai",
    type="language",
    credential=str(cred[0].id),
)
await model.save()

Summary

• Podcast profile migration runs automatically on API startup after schema migrations complete
• Idempotent design skips records that already contain Model references, ensuring safe re-runs
• Dual-table processing upgrades both episode_profile (for LLM configs) and speaker_profile (for voice models)
• Credential linkage ties new Model records to existing provider credentials via Credential.get_by_provider()
• Repository pattern uses repo_update and ensure_record_id for database operations

Frequently Asked Questions

What triggers the podcast profile migration?

The migration triggers automatically when the API server starts. The migrate_podcast_profiles() function in open_notebook/podcasts/migration.py is called from api/main.py immediately after SQL schema migrations finish executing.

How does the migration handle missing Model records?

When _find_model_record() returns no match, _find_or_create_model() attempts to locate a credential for the provider using Credential.get_by_provider(). If found, it creates a new Model instance, persists it to the database, and returns the new record ID for assignment to the profile.

Is the migration safe to run multiple times?

Yes. The migration is fully idempotent. It checks whether outline_llm, transcript_llm, or voice_model fields already contain values before processing any row. Populated fields cause the migration to skip that specific record, preventing duplicate processing or data corruption.

Which legacy fields get converted during the migration?

For EpisodeProfile records, the legacy outline_provider/outline_model and transcript_provider/transcript_model string pairs convert to outline_llm and transcript_llm record references. For SpeakerProfile records, the tts_provider/tts_model pair converts to the voice_model field.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}