How Multi-Speaker Podcast Generation Works with EpisodeProfiles in Open Notebook

Multi-speaker podcast generation in Open Notebook combines EpisodeProfiles for episode configuration with SpeakerProfiles for voice definitions, orchestrated through an asynchronous background job that resolves model configurations, generates outlines and transcripts, and synthesizes audio per speaker.

The lfnovo/open-notebook repository implements a sophisticated podcast generation system that separates content structure from speaker identity. This architecture allows you to mix and match different episode formats with various speaker combinations while maintaining clean configuration management through SurrealDB-backed profiles.

Understanding EpisodeProfiles and SpeakerProfiles

The system uses two distinct profile types to decouple episode metadata from speaker definitions.

EpisodeProfiles define the overall episode configuration stored in open_notebook/podcasts/models.py. These include the outline LLM, transcript LLM, language settings, segment count, and default briefing template. The EpisodeProfile.get_by_name method retrieves these configurations from the database.

SpeakerProfiles handle one-to-four speaker definitions, each with its own voice model and optional per-speaker overrides. Retrieved via SpeakerProfile.get_by_name, these profiles contain persona details like backstory, personality traits, and voice identifiers that shape how each speaker sounds and behaves during the episode.

The Generation Pipeline

Multi-speaker podcast generation runs as an asynchronous background job through a ten-step process defined in commands/podcast_commands.py.

Job Submission and Validation

The process begins when the API receives a PodcastGenerationRequest containing the episode profile name, speaker profile name, episode title, and optional content sources. The PodcastService.submit_generation_job method in api/podcast_service.py validates that both profiles exist and creates a surreal-commands job via submit_command.

from open_notebook.api.podcast_service import PodcastService

job_id = await PodcastService.submit_generation_job(
    episode_profile_name="TechDebate",
    speaker_profile_name="ExpertPanel",
    episode_name="AI-Safety Debate",
    notebook_id="notebook:12345",
    briefing_suffix="Focus on recent papers from 2024."
)

Profile Loading and Model Resolution

Once the background command generate_podcast executes, it loads the selected profiles. The system resolves abstract model references to concrete provider/model/config triples using _resolve_model_config. This occurs for the episode's outline_llm and transcript_llm fields, as well as the speaker profile's voice_model and any per-speaker voice_model overrides.

# Episode profile example (JSON stored in SurrealDB)

{
  "name": "TechDebate",
  "speaker_config": "ExpertPanel",
  "outline_llm": "model:openai:gpt-4o",
  "transcript_llm": "model:openai:gpt-4o-mini",
  "language": "en-US",
  "default_briefing": "Discuss recent advances in AI safety.",
  "num_segments": 6
}

Configuration Injection and Briefing

The command converts all database profiles into dictionaries and enriches them with resolved configuration values. Faulty profiles are pruned to ensure the podcast-creator pipeline receives validated data. Episode configuration enrichment occurs at lines 49-71, while speaker configuration enrichment happens at lines 75-93 in commands/podcast_commands.py.

The briefing combines the episode profile's default_briefing with an optional user-supplied briefing_suffix:

# Briefing creation (lines 110-114)

briefing = f"{episode_profile.default_briefing}\n\n{briefing_suffix}"

Audio Generation and Persistence

The system creates a PodcastEpisode record that snapshots the fully-resolved profiles, ensuring later edits don't affect generated episodes. A UUID-named directory under DATA_FOLDER/podcasts/episodes/ is created via build_episode_output_dir to avoid filesystem issues with special characters.

The create_podcast function from the podcast-creator pipeline executes the actual generation:

1. Generates an outline using the resolved outline LLM
2. Generates a transcript using the resolved transcript LLM
3. Synthesizes each speaker's dialogue using their respective TTS models
4. Mixes audio streams into a final MP3

Results including final_output_file_path, transcript, and outline are written back to the PodcastEpisode record at lines 244-262.

Working with Speaker Profiles

Speaker profiles support granular voice control through the voice_model field and per-speaker overrides.

{
  "name": "ExpertPanel",
  "voice_model": "model:elevenlabs:en_us_male",
  "speakers": [
    {
      "name": "Dr. Ada",
      "voice_id": "female",
      "backstory": "AI safety researcher",
      "personality": "curious, thorough",
      "voice_model": "model:elevenlabs:en_us_female"
    },
    {
      "name": "Prof. Turing",
      "voice_id": "male",
      "backstory": "Systems engineer",
      "personality": "dry humor, skeptical"
    }
  ]
}

In this configuration, Dr. Ada uses the per-speaker voice_model override, while Prof. Turing falls back to the profile-level voice_model.

Monitoring Generation Status

Clients poll job status through PodcastService.get_job_status, which forwards to surreal_commands.get_command_status:

status = await PodcastService.get_job_status(job_id)

# Returns: {'job_id': '...', 'status': 'running', ...}

Summary

• EpisodeProfiles control episode structure, language, segment count, and LLM selection stored in open_notebook/podcasts/models.py
• SpeakerProfiles define 1-4 speakers with individual voice models and personality attributes
• The generation pipeline runs asynchronously via commands/podcast_commands.py with model resolution occurring at lines 13-29
• Per-speaker voice model overrides take precedence over profile-level defaults
• Generated episodes snapshot profile configurations to prevent mutation of historical records
• Filesystem-safe UUID directories prevent special character issues in episode titles

Frequently Asked Questions

What is the relationship between EpisodeProfiles and SpeakerProfiles?

EpisodeProfiles and SpeakerProfiles maintain a loose coupling in the Open Notebook architecture. An EpisodeProfile references a SpeakerProfile through its speaker_config field, allowing you to reuse the same speaker ensemble across different episode formats. The generate_podcast command loads both profiles independently and validates their existence before processing.

How does Open Notebook handle different voice models per speaker?

The system resolves voice models through _resolve_model_config for each speaker, checking for per-speaker voice_model overrides first, then falling back to the SpeakerProfile's default voice_model. Each speaker's dialogue is synthesized separately using their resolved TTS configuration before audio mixing occurs.

Can I override models for individual speakers?

Yes. While the EpisodeProfile defines global outline_llm and transcript_llm settings, individual speakers within a SpeakerProfile can specify their own voice_model overrides. These per-speaker configurations take precedence over the profile-level voice model during the resolution phase at lines 75-93 of commands/podcast_commands.py.

Where are generated podcast files stored?

Generated episodes are stored in UUID-named directories under DATA_FOLDER/podcasts/episodes/, created via build_episode_output_dir to eliminate filesystem issues with special characters in episode titles. The final MP3 path, along with the transcript and outline JSON, is persisted in the PodcastEpisode database record for later retrieval.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}