# Job Queue Architecture for Async Podcast Generation Using Surreal-Commands in Open Notebook

> Explore the job queue architecture for async podcast generation in Open Notebook. Learn how Surreal-Commands store RecordIDs for status polling, result retrieval, and retries.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: architecture
- Published: 2026-06-18

---

**Open Notebook leverages Surreal-Commands as a distributed task queue to handle podcast generation asynchronously, storing the command RecordID on the PodcastEpisode model to enable status polling, result retrieval, and clean retry operations.**

Open Notebook implements a robust job queue architecture for async podcast generation by integrating Surreal-Commands, a distributed task processing system. This design decouples the computationally intensive audio synthesis pipeline from the FastAPI request layer, allowing users to submit generation requests and monitor progress through persistent job identifiers stored on `PodcastEpisode` records.

## Submitting Generation Jobs to the Queue

The async workflow begins when a client sends a `POST` request to `/podcasts/generate`. In [`api/podcast_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/podcast_service.py), the `PodcastService.submit_generation_job` method validates the requested profiles and constructs a command argument dictionary. It dynamically imports `commands.podcast_commands` to ensure the command is visible to the Surreal-Commands registry, then invokes `submit_command("open_notebook", "generate_podcast", args)`.

This function returns a SurrealDB `RecordID` that is immediately stored as a string in the `job_id` field and returned to the client. This establishes the initial link between the API request and the background job queue, allowing the client to begin polling for status while the heavy processing moves to the background worker.

## Command Execution and Episode Persistence

When Surreal-Commands picks up the queued task, it executes the `generate_podcast_command` function defined in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py). The function is decorated with `@command("generate_podcast", app="open_notebook", …)` to register it with the command framework.

Inside the command execution, the code creates a `PodcastEpisode` model and stores the Surreal-Commands `command_id` on the episode's `command` field using `ensure_record_id(input_data.execution_context.command_id)`. Crucially, the episode record is persisted **before** the heavyweight podcast-creator pipeline begins processing. This ensures the job remains trackable even if the process crashes during the long-running audio generation phase.

## Persisting the Job Link with RecordID

The connection between the episode and its background job relies on the `command` field in [`open_notebook/podcasts/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/podcasts/models.py). This field stores a `RecordID` that points back to the Surreal-Commands job record.

To maintain data integrity, the `PodcastEpisode` model overrides `_prepare_save_data` to convert the `command` field to a proper `RecordID` instance before writing to SurrealDB. This guarantees that the foreign key relationship remains valid across queries and status checks, preventing serialization mismatches when the episode is retrieved or updated.

## Monitoring Job Status and Results

Clients monitor generation progress by calling `GET /podcasts/jobs/{job_id}`, handled by the `get_podcast_job_status` router in [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py). This endpoint delegates to `PodcastService.get_job_status`, which simply awaits `get_command_status(job_id)` from the Surreal-Commands library.

The response includes the raw status (`submitted`, `running`, `completed`, `failed`), result payload, timestamps, and error messages. When listing episodes, the same `get_job_detail()` helper method embeds current job status alongside episode metadata, allowing the UI to display real-time generation progress without requiring separate API calls.

## Retry and Cleanup Mechanisms

If a job fails, the system provides a clean retry path through `POST /podcasts/episodes/{episode_id}/retry`. This endpoint, defined in [`api/routers/podcasts.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/podcasts.py), deletes the failed episode record—including any associated audio files—from storage, then re-submits a fresh generation request via `PodcastService.submit_generation_job`.

This approach guarantees a clean slate by generating a new Surreal-Commands `RecordID` rather than attempting to resurrect a corrupted or stale job state. This ensures deterministic retry behavior and prevents contamination from previous failed attempts.

## Implementation Examples

### Submitting a Podcast Generation Job

```python
import httpx

payload = {
    "episode_profile": "my-episode-profile",
    "speaker_profile": "default-speaker",
    "episode_name": "TheFutureOfAI",
    "notebook_id": "notebook-123",
    "briefing_suffix": "Add a short intro."
}

resp = httpx.post("http://localhost:5055/podcasts/generate", json=payload)
job = resp.json()
print(f"Job submitted – ID: {job['job_id']}")

```

### Polling for Job Completion

```python
import time, httpx

job_id = "job:open_notebook:generate_podcast:0001"
while True:
    r = httpx.get(f"http://localhost:5055/podcasts/jobs/{job_id}")
    data = r.json()
    print(f"Status: {data['status']}")
    if data["status"] in ("completed", "failed"):
        break
    time.sleep(2)

```

### Retrying a Failed Episode

```python
import httpx

failed_episode = "episode:bad123"
r = httpx.post(f"http://localhost:5055/podcasts/episodes/{failed_episode}/retry")
print(r.json())

```

## Summary

- **Surreal-Commands Integration**: Open Notebook uses `submit_command` and `get_command_status` from the Surreal-Commands library to queue and monitor podcast generation tasks asynchronously.
- **Early Persistence**: The `PodcastEpisode` record is created and saved before audio processing begins, storing the `command_id` to maintain job visibility even during process crashes.
- **Status Polling**: The `GET /podcasts/jobs/{job_id}` endpoint provides real-time access to job states including `submitted`, `running`, `completed`, and `failed`.
- **Clean Retries**: Failed jobs trigger deletion of the old episode and submission of a fresh Surreal-Commands job, ensuring no stale state persists.
- **RecordID Management**: The `command` field uses `_prepare_save_data` to ensure proper `RecordID` serialization when persisting to SurrealDB.

## Frequently Asked Questions

### How does Open Notebook link a PodcastEpisode to its Surreal-Commands job?

The `PodcastEpisode` model stores the Surreal-Commands `RecordID` in its `command` field. When the `generate_podcast_command` executes in [`commands/podcast_commands.py`](https://github.com/lfnovo/open-notebook/blob/main/commands/podcast_commands.py), it captures the `command_id` from the execution context and stores it using `ensure_record_id()`. The model's `_prepare_save_data` method ensures this value is properly serialized as a `RecordID` when saving to SurrealDB.

### What happens if the podcast generation process crashes midway?

Because the `PodcastEpisode` record is persisted **before** the long-running audio synthesis begins, the job remains trackable even if the process crashes. The episode exists in the database with its `command` field populated, allowing the system to query the job status (which would show as `failed` or remain `running` depending on the crash timing) and enabling users to retry via the dedicated endpoint.

### How does the retry mechanism ensure clean job state?

When retrying a failed episode via `POST /podcasts/episodes/{episode_id}/retry`, the endpoint first deletes the failed episode record and any associated audio files from storage. It then calls `PodcastService.submit_generation_job` to create a completely new Surreal-Commands job with a fresh `RecordID`, preventing contamination from the previous failed attempt and ensuring deterministic execution.

### Can I check the status of a podcast generation job without querying the episode record?

Yes. The `GET /podcasts/jobs/{job_id}` endpoint allows direct status queries by calling `get_command_status(job_id)` from the Surreal-Commands library. This returns the raw job status, timestamps, and error messages without requiring you to fetch the full episode record, though the episode list view also embeds this status via the `get_job_detail()` helper method.