How Multi-Speaker Podcast Generation Works with Custom Profiles in Open Notebook

Multi-speaker podcast generation in Open Notebook uses a profile-driven pipeline where SpeakerProfile defines up to 4 speakers with distinct voice IDs and TTS models, while EpisodeProfile orchestrates content generation through the podcast-creator library.

Open Notebook (lfnovo/open-notebook) implements declarative podcast creation through specialized profile models stored in SurrealDB. The system separates speaker configuration from episode generation logic, allowing you to reuse voice personalities across multiple podcast episodes while mixing different text-to-speech (TTS) providers within a single audio file.

Profile Architecture: SpeakerProfile and EpisodeProfile

The foundation of multi-speaker generation resides in open_notebook/podcasts/models.py, which defines two primary configuration models.

SpeakerProfile supports 1-4 speakers through a structured speakers list. Each speaker entry requires four fields:

• name – The speaker's identifier used in transcripts
• voice_id – The specific voice identifier for the TTS provider
• backstory – Context shaping the speaker's domain knowledge
• personality – Behavioral traits influencing dialogue style

The profile also stores a global voice_model referencing an Esperanto model record (e.g., "model:tts/openai/tts-1"), with optional per-speaker overrides via a voice_model key inside individual speaker dictionaries.

EpisodeProfile defines the generation parameters including outline_llm, transcript_llm, and references to the global voice_model. These profiles work together to create a complete podcast configuration that the generation command consumes.

Model Resolution with _resolve_model_config

Before generation begins, the system resolves credential and model configurations through the helper function _resolve_model_config() in open_notebook/podcasts/models.py.

This function:

1. Loads the Esperanto model record from the database
2. Merges stored credential configurations
3. Falls back to provider-wide defaults when specific credentials are absent

The resolver handles both the episode-level models (outline and transcript generation) and per-speaker TTS models, ensuring API keys and provider settings are correctly injected before the podcast-creator library receives the configuration.

The Generation Pipeline

Located in commands/podcast_commands.py, the generate_podcast_command orchestrates the entire multi-speaker workflow.

The command executes these critical steps:

1. Profile Loading – Retrieves the requested EpisodeProfile and linked SpeakerProfile from SurrealDB
2. Validation – Confirms that outline_llm, transcript_llm, and voice_model fields are present
3. Model Resolution – Calls _resolve_model_config for both the episode's AI models and each speaker's TTS configuration
4. Dictionary Construction – Iterates over every stored EpisodeProfile and SpeakerProfile to pre-populate configuration dictionaries for the podcast-creator library (lines 95-105 in commands/podcast_commands.py)
5. Per-Speaker Override Resolution – During iteration, detects and applies individual speaker TTS overrides, allowing one speaker to use OpenAI's tts-1 while another uses tts-1-mini
6. Configuration Injection – Passes the collected data to podcast-creator via configure("episode_config", …) and configure("speakers_config", …)

The podcast-creator library expects a speakers configuration map where each entry specifies tts_provider, tts_model, and optional tts_config. By populating this map with resolved model data from Open Notebook, the system enables a single podcast episode to narrate different segments with distinct voices and potentially different TTS engines.

Persisting Episodes and Job Tracking

After configuration, the system creates a PodcastEpisode record (also defined in open_notebook/podcasts/models.py) that stores:

• References to the chosen EpisodeProfile and SpeakerProfile
• The generated briefing and transcript content
• The final audio file path
• The SurrealDB record ID of the running background job

This persistence allows the FastAPI layer to query generation status asynchronously, separating the long-running TTS synthesis from the HTTP request lifecycle.

API Integration and Endpoints

The FastAPI router in api/routers/podcasts.py exposes three primary endpoints for multi-speaker generation:

• POST /podcasts/generate – Accepts episode_profile, speaker_profile, notebook_id, and content, then submits a background job (returns immediately with a job_id)
• GET /podcasts/episodes – Lists all episodes with their current generation status and audio URLs
• GET /podcasts/episodes/{id} – Retrieves complete metadata for a specific episode including speaker configuration and playback links

The service layer in api/podcast_service.py bridges these HTTP endpoints to the underlying generate_podcast_command, handling job submission, status lookup, and episode CRUD operations.

Creating Custom Speaker Profiles

To define a multi-speaker configuration with per-speaker TTS overrides:

from open_notebook.podcasts.models import SpeakerProfile

await SpeakerProfile(
    name="roundtable_demo",
    description="A round-table discussion with three distinct voices",
    voice_model="model:tts/openai/tts-1",  # Global fallback

    speakers=[
        {
            "name": "Alice",
            "voice_id": "female_en_us_1",
            "backstory": "Tech journalist",
            "personality": "Curious, analytical",
            "voice_model": "model:tts/openai/tts-1-mini",  # Per-speaker override

        },
        {
            "name": "Bob",
            "voice_id": "male_en_us_2",
            "backstory": "Seasoned engineer",
            "personality": "Pragmatic, methodical",
        },
        {
            "name": "Cara",
            "voice_id": "female_en_uk_1",
            "backstory": "Product designer",
            "personality": "Creative, enthusiastic",
        },
    ],
).save()

Trigger generation via the REST API:

curl -X POST http://localhost:5055/api/podcasts/generate \
  -H "Content-Type: application/json" \
  -d '{
        "episode_profile": "daily_briefing",
        "speaker_profile": "roundtable_demo",
        "episode_name": "Tech Trends Sep 2026",
        "notebook_id": "notebook:my-research",
        "content": "Full text of the research notes …",
        "briefing_suffix": "Emphasise the impact of AI on education."
      }'

Poll for completion using the returned job_id:

curl http://localhost:5055/api/podcasts/jobs/<job_id>

Retrieve the final audio:

curl http://localhost:5055/api/podcasts/episodes/<episode_id>

Summary

• Open Notebook implements multi-speaker podcast generation through profile-driven architecture separating SpeakerProfile (voice/personality) from EpisodeProfile (content generation settings)
• SpeakerProfile supports 1-4 speakers with required fields: name, voice_id, backstory, and personality, plus optional per-speaker voice_model overrides
• _resolve_model_config() in open_notebook/podcasts/models.py handles credential resolution and model configuration merging for both LLM and TTS providers
• generate_podcast_command in commands/podcast_commands.py validates profiles, resolves models, builds configuration dictionaries, and injects them into the podcast-creator library via configure() calls
• The API layer in api/routers/podcasts.py provides asynchronous job submission with background processing status tracking through SurrealDB job records

Frequently Asked Questions

How many speakers can a single podcast episode include?

Open Notebook supports 1 to 4 speakers per SpeakerProfile as defined in open_notebook/podcasts/models.py. Each speaker requires a unique name, voice_id, backstory, and personality definition. The system validates these constraints during profile creation, ensuring the podcast-creator library receives a compatible speakers configuration map.

Can different speakers use different text-to-speech providers in the same episode?

Yes. While the SpeakerProfile defines a global voice_model fallback, individual speakers can override this via a voice_model field in their configuration dictionary. The generate_podcast_command resolves these overrides separately (lines 95-105 in commands/podcast_commands.py), allowing Speaker A to use OpenAI's TTS while Speaker B uses a different provider or model entirely.

Where does Open Notebook store the generated podcast audio files?

The PodcastEpisode model in open_notebook/podcasts/models.py stores the audio file path alongside the transcript, outline, and profile references. The api/routers/podcasts.py endpoint serves these files via HTTP, with the episode record linking the SurrealDB job ID to the final audio asset for status tracking and retrieval.

What happens if a speaker's voice model configuration is invalid?

The _resolve_model_config() function attempts to load the specified Esperanto model record and merge credentials. If the model reference is invalid or credentials are missing, the resolution fails before the podcast-creator library receives the configuration. This validation occurs during the generate_podcast_command execution, preventing invalid TTS requests from consuming API resources.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}