# Implementing Multi-Speaker Podcast Generation in Open-Notebook: A Complete Technical Guide

> Learn to implement multi-speaker podcast generation using Open-Notebook. This guide details voice model orchestration with SurrealDB and podcast-creator library.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-30

---

**Open-Notebook supports fully-featured multi-speaker podcast generation by orchestrating distinct voice models per speaker through SurrealDB-backed profiles and the podcast-creator library.**

The `lfnovo/open-notebook` repository provides a production-ready architecture for implementing multi-speaker podcast generation, enabling dynamic audio synthesis with up to four distinct speakers per episode. This system decouples speaker configuration from episode logic, allowing each participant to utilize custom text-to-speech (TTS) models while maintaining a unified generation pipeline.

## Architecture Overview

The multi-speaker implementation spans four distinct layers: domain models for data persistence, a service layer for job orchestration, command handlers for background processing, and REST endpoints for client interaction.

### Domain Models

In [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py), three primary classes define the data structures stored in SurrealDB:

- **EpisodeProfile** – Links to a speaker configuration via `speaker_config` and stores model IDs for outline and transcript generation.
- **SpeakerProfile** – Contains a `speakers` array supporting up to four entries. Each speaker definition includes `name`, `voice_id`, `backstory`, `personality`, and an optional `voice_model` field that overrides the profile-wide TTS model.
- **PodcastEpisode** – Records episode metadata, the associated command ID, and generated assets including the audio file path and transcript.

### Service Layer

The [`api/podcast_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/podcast_service.py) file implements `PodcastService`, which coordinates job submission and status tracking:

- `submit_generation_job` validates requested profiles, resolves notebook content when supplied, and queues the background job via `surreal-commands`.
- `list_episodes`, `get_episode`, and `get_job_status` provide read-only APIs consumed by the frontend to display generation progress.

### Command Implementation

Heavy-weight processing occurs in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py) within the `generate_podcast_command` function. This worker:

1. Loads the selected `EpisodeProfile` and its linked `SpeakerProfile`.
2. Resolves all model IDs (outline, transcript, TTS) to concrete provider/model/config triples using `_resolve_model_config`.
3. Mass-converts episode and speaker profiles to the JSON format expected by the **podcast-creator** library.
4. Iterates over `SpeakerProfile.speakers` to attach individual `voice_model` configurations when specified, enabling per-speaker TTS selection.

### API Router

HTTP endpoints in [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py) expose the service to clients:

- `POST /podcasts/generate` triggers `submit_generation_job` and returns a job ID.
- `GET /podcasts/episodes` returns episode lists with `job_status` and streaming URLs.
- `GET /podcasts/episodes/{episode_id}/audio` streams the final MP3 file.

## Configuring Multi-Speaker Profiles

Implementing multi-speaker synthesis requires defining speaker configurations before episode generation.

### Creating a SpeakerProfile

The `SpeakerProfile` class stores the `speakers` array where each entry contains:

- `name` – Human-readable identifier (e.g., "Narrator", "Expert").
- `voice_id`, `backstory`, `personality` – Metadata guiding the LLM to maintain distinct voices.
- `voice_model` – Optional Model record ID enabling per-speaker TTS overrides.

When `voice_model` is omitted for a specific speaker, the system falls back to the profile-wide `voice_model` defined in the `SpeakerProfile` instance.

### Linking to EpisodeProfile

The `EpisodeProfile.speaker_config` field stores the name of the `SpeakerProfile` to use. During generation, the command worker fetches this profile and transforms its speakers into the JSON structure required by the **podcast-creator** library.

## The Generation Pipeline

When you submit a generation request, the system executes an asynchronous pipeline that transforms text content into multi-speaker audio.

### Job Submission and Validation

Calling `PodcastService.submit_generation_job` performs the following:

1. Validates that the requested `episode_profile` and `speaker_profile` exist in SurrealDB.
2. Resolves the notebook content if `notebook_id` is provided in the request.
3. Creates a command record and queues it for background processing.

### Background Processing with podcast-creator

The `generate_podcast_command` worker executes the synthesis:

1. **Outline Generation** – Uses the `outline_llm` specified in `EpisodeProfile` to create a structured episode plan.
2. **Transcript Generation** – Generates the full conversational script using the `transcript_llm`, applying speaker definitions to maintain distinct voices.
3. **Multi-Speaker Audio Synthesis** – Splits the transcript into chunks per speaker, resolves individual TTS models (honoring `voice_model` overrides), and concatenates the resulting audio streams into a single MP3.

Because each speaker can declare its own `voice_model`, the system can mix voices (e.g., OpenAI's "alloy" for a host and ElevenLabs' English Female V2 for a guest) without code modifications.

### Audio Streaming and Retrieval

Upon completion, the episode record updates with `audio_url` pointing to the generated file. Clients retrieve this via `GET /podcasts/episodes/{episode_id}/audio`, which streams the MP3 directly from storage.

## Implementation Examples

### 1. Define a Multi-Speaker Profile

```python
from open_notebook.podcasts.models import SpeakerProfile

await SpeakerProfile(
    name="InterviewShow",
    description="Host + Guest conversation",
    voice_model="model:openai:gpt-4o-mini",
    speakers=[
        {
            "name": "Host",
            "voice_id": "alloy",
            "backstory": "A seasoned tech journalist.",
            "personality": "Friendly, inquisitive",
            # Uses the profile-wide TTS model

        },
        {
            "name": "Guest",
            "voice_id": "fable",
            "backstory": "AI researcher with a British accent.",
            "personality": "Thoughtful, measured",
            "voice_model": "model:elevenlabs:eleven_english_female_v2"
        },
    ],
).save()

```

*Source: [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py)*

### 2. Create an Episode Profile

```python
from open_notebook.podcasts.models import EpisodeProfile

await EpisodeProfile(
    name="AI_Trend_Episode",
    description="Weekly AI trends roundup",
    speaker_config="InterviewShow",
    outline_llm="model:openai:gpt-4o-mini",
    transcript_llm="model:openai:gpt-4o",
    default_briefing="Summarize recent AI research and let the Host interview the Guest.",
    num_segments=6,
).save()

```

*Source: [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py)*

### 3. Submit a Generation Job

```python
import httpx

payload = {
    "episode_profile": "AI_Trend_Episode",
    "speaker_profile": "InterviewShow",
    "episode_name": "AI Trends – Week 42",
    "notebook_id": "notebook:my_notebook",
    "briefing_suffix": "Include a short outro with a call-to-action."
}

resp = httpx.post("http://localhost:5055/podcasts/generate", json=payload)
job = resp.json()
print("Job submitted:", job["job_id"])

```

*Source: [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py)*

### 4. Poll for Completion

```python
import time
import httpx

job_id = job["job_id"]
while True:
    status = httpx.get(f"http://localhost:5055/podcasts/jobs/{job_id}").json()
    print(status["status"])
    if status["status"] in ("completed", "failed"):
        break
    time.sleep(5)

```

*Source: [`api/podcast_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/podcast_service.py)*

### 5. Retrieve the Audio URL

```python
episodes = httpx.get("http://localhost:5055/podcasts/episodes").json()
for ep in episodes:
    if ep["name"] == "AI Trends – Week 42":
        print("Audio URL:", ep["audio_url"])
        break

```

*Source: [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py)*

## Summary

- **Multi-speaker support** is implemented through the `SpeakerProfile` domain model in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py), which stores up to four speaker definitions per profile.
- **Per-speaker voice customization** occurs via the optional `voice_model` field, allowing individual TTS models that override the profile default.
- **Asynchronous generation** is handled by `generate_podcast_command` in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py), which orchestrates the **podcast-creator** library.
- **Job lifecycle management** is provided by `PodcastService` in [`api/podcast_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/podcast_service.py), handling submission, validation, and status polling.
- **Audio delivery** is exposed through REST endpoints in [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py), supporting both metadata retrieval and MP3 streaming.

## Frequently Asked Questions

### How many speakers does Open-Notebook support per podcast episode?

Open-Notebook supports up to four speakers per episode, defined as an array within the `SpeakerProfile` model. Each speaker requires a unique name, voice characteristics, and optional TTS model configuration.

### Can each speaker use a different text-to-speech model?

Yes. While `SpeakerProfile` defines a default `voice_model` for the entire profile, individual speakers can override this by specifying their own `voice_model` record ID. This enables mixing providers (e.g., OpenAI for one speaker, ElevenLabs for another) within a single episode.

### What happens if a speaker doesn't specify a voice_model?

If a speaker entry lacks a `voice_model` field, the system automatically falls back to the profile-wide `voice_model` defined in the parent `SpeakerProfile` instance. If neither is specified, the generation command will fail during model resolution in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py).

### How do I monitor the status of a podcast generation job?

Submit a `POST` request to `/podcasts/generate` to receive a job ID, then poll `GET /podcasts/jobs/{job_id}` to retrieve status updates. The `PodcastService.get_job_status` method in [`api/podcast_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/podcast_service.py) returns states including "pending", "running", "completed", or "failed", allowing clients to track progress until the audio file becomes available.