# How Multi-Speaker Podcast Generation Works with Custom Profiles in Open Notebook

> Discover how multi-speaker podcast generation in Open Notebook uses custom profiles to create dynamic audio content with distinct voices. Learn about the SpeakerProfile and EpisodeProfile pipeline.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-15

---

**Multi-speaker podcast generation in Open Notebook uses a profile-driven pipeline where `SpeakerProfile` defines up to 4 speakers with distinct voice IDs and TTS models, while `EpisodeProfile` orchestrates content generation through the `podcast-creator` library.**

Open Notebook (lfnovo/open-notebook) implements declarative podcast creation through specialized profile models stored in SurrealDB. The system separates speaker configuration from episode generation logic, allowing you to reuse voice personalities across multiple podcast episodes while mixing different text-to-speech (TTS) providers within a single audio file.

## Profile Architecture: SpeakerProfile and EpisodeProfile

The foundation of multi-speaker generation resides in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py), which defines two primary configuration models.

**SpeakerProfile** supports **1-4 speakers** through a structured `speakers` list. Each speaker entry requires four fields:
- `name` – The speaker's identifier used in transcripts
- `voice_id` – The specific voice identifier for the TTS provider
- `backstory` – Context shaping the speaker's domain knowledge
- `personality` – Behavioral traits influencing dialogue style

The profile also stores a **global `voice_model`** referencing an Esperanto model record (e.g., `"model:tts/openai/tts-1"`), with optional **per-speaker overrides** via a `voice_model` key inside individual speaker dictionaries.

**EpisodeProfile** defines the generation parameters including `outline_llm`, `transcript_llm`, and references to the global `voice_model`. These profiles work together to create a complete podcast configuration that the generation command consumes.

## Model Resolution with `_resolve_model_config`

Before generation begins, the system resolves credential and model configurations through the helper function `_resolve_model_config()` in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py).

This function:
1. Loads the Esperanto model record from the database
2. Merges stored credential configurations
3. Falls back to provider-wide defaults when specific credentials are absent

The resolver handles both the episode-level models (outline and transcript generation) and per-speaker TTS models, ensuring API keys and provider settings are correctly injected before the `podcast-creator` library receives the configuration.

## The Generation Pipeline

Located in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py), the `generate_podcast_command` orchestrates the entire multi-speaker workflow.

The command executes these critical steps:

1. **Profile Loading** – Retrieves the requested `EpisodeProfile` and linked `SpeakerProfile` from SurrealDB
2. **Validation** – Confirms that `outline_llm`, `transcript_llm`, and `voice_model` fields are present
3. **Model Resolution** – Calls `_resolve_model_config` for both the episode's AI models and each speaker's TTS configuration
4. **Dictionary Construction** – Iterates over every stored `EpisodeProfile` and `SpeakerProfile` to pre-populate configuration dictionaries for the `podcast-creator` library (lines 95-105 in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py))
5. **Per-Speaker Override Resolution** – During iteration, detects and applies individual speaker TTS overrides, allowing one speaker to use OpenAI's `tts-1` while another uses `tts-1-mini`
6. **Configuration Injection** – Passes the collected data to `podcast-creator` via `configure("episode_config", …)` and `configure("speakers_config", …)`

The `podcast-creator` library expects a **speakers configuration map** where each entry specifies `tts_provider`, `tts_model`, and optional `tts_config`. By populating this map with resolved model data from Open Notebook, the system enables a single podcast episode to narrate different segments with distinct voices and potentially different TTS engines.

## Persisting Episodes and Job Tracking

After configuration, the system creates a `PodcastEpisode` record (also defined in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py)) that stores:
- References to the chosen `EpisodeProfile` and `SpeakerProfile`
- The generated briefing and transcript content
- The final audio file path
- The SurrealDB record ID of the running background job

This persistence allows the FastAPI layer to query generation status asynchronously, separating the long-running TTS synthesis from the HTTP request lifecycle.

## API Integration and Endpoints

The FastAPI router in [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py) exposes three primary endpoints for multi-speaker generation:

- **POST** `/podcasts/generate` – Accepts `episode_profile`, `speaker_profile`, `notebook_id`, and content, then submits a background job (returns immediately with a `job_id`)
- **GET** `/podcasts/episodes` – Lists all episodes with their current generation status and audio URLs
- **GET** `/podcasts/episodes/{id}` – Retrieves complete metadata for a specific episode including speaker configuration and playback links

The service layer in [`api/podcast_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/podcast_service.py) bridges these HTTP endpoints to the underlying `generate_podcast_command`, handling job submission, status lookup, and episode CRUD operations.

## Creating Custom Speaker Profiles

To define a multi-speaker configuration with per-speaker TTS overrides:

```python
from open_notebook.podcasts.models import SpeakerProfile

await SpeakerProfile(
    name="roundtable_demo",
    description="A round-table discussion with three distinct voices",
    voice_model="model:tts/openai/tts-1",  # Global fallback

    speakers=[
        {
            "name": "Alice",
            "voice_id": "female_en_us_1",
            "backstory": "Tech journalist",
            "personality": "Curious, analytical",
            "voice_model": "model:tts/openai/tts-1-mini",  # Per-speaker override

        },
        {
            "name": "Bob",
            "voice_id": "male_en_us_2",
            "backstory": "Seasoned engineer",
            "personality": "Pragmatic, methodical",
        },
        {
            "name": "Cara",
            "voice_id": "female_en_uk_1",
            "backstory": "Product designer",
            "personality": "Creative, enthusiastic",
        },
    ],
).save()

```

Trigger generation via the REST API:

```bash
curl -X POST http://localhost:5055/api/podcasts/generate \
  -H "Content-Type: application/json" \
  -d '{
        "episode_profile": "daily_briefing",
        "speaker_profile": "roundtable_demo",
        "episode_name": "Tech Trends Sep 2026",
        "notebook_id": "notebook:my-research",
        "content": "Full text of the research notes …",
        "briefing_suffix": "Emphasise the impact of AI on education."
      }'

```

Poll for completion using the returned `job_id`:

```bash
curl http://localhost:5055/api/podcasts/jobs/<job_id>

```

Retrieve the final audio:

```bash
curl http://localhost:5055/api/podcasts/episodes/<episode_id>

```

## Summary

- **Open Notebook** implements multi-speaker podcast generation through profile-driven architecture separating `SpeakerProfile` (voice/personality) from `EpisodeProfile` (content generation settings)
- **SpeakerProfile** supports 1-4 speakers with required fields: `name`, `voice_id`, `backstory`, and `personality`, plus optional per-speaker `voice_model` overrides
- **`_resolve_model_config()`** in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py) handles credential resolution and model configuration merging for both LLM and TTS providers
- **`generate_podcast_command`** in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py) validates profiles, resolves models, builds configuration dictionaries, and injects them into the `podcast-creator` library via `configure()` calls
- **The API layer** in [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py) provides asynchronous job submission with background processing status tracking through SurrealDB job records

## Frequently Asked Questions

### How many speakers can a single podcast episode include?

Open Notebook supports **1 to 4 speakers** per `SpeakerProfile` as defined in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py). Each speaker requires a unique `name`, `voice_id`, `backstory`, and `personality` definition. The system validates these constraints during profile creation, ensuring the `podcast-creator` library receives a compatible speakers configuration map.

### Can different speakers use different text-to-speech providers in the same episode?

Yes. While the `SpeakerProfile` defines a global `voice_model` fallback, individual speakers can override this via a `voice_model` field in their configuration dictionary. The `generate_podcast_command` resolves these overrides separately (lines 95-105 in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py)), allowing Speaker A to use OpenAI's TTS while Speaker B uses a different provider or model entirely.

### Where does Open Notebook store the generated podcast audio files?

The `PodcastEpisode` model in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py) stores the audio file path alongside the transcript, outline, and profile references. The [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py) endpoint serves these files via HTTP, with the episode record linking the SurrealDB job ID to the final audio asset for status tracking and retrieval.

### What happens if a speaker's voice model configuration is invalid?

The `_resolve_model_config()` function attempts to load the specified Esperanto model record and merge credentials. If the model reference is invalid or credentials are missing, the resolution fails before the `podcast-creator` library receives the configuration. This validation occurs during the `generate_podcast_command` execution, preventing invalid TTS requests from consuming API resources.