How Spotify-Saver Handles Concurrent Downloads with Progress Callbacks: Async Architecture Explained

The DownloadService leverages Python's async/await pattern combined with thread-pool offloading to execute multiple blocking yt-dlp downloads in parallel, while propagating real-time progress updates through a lightweight synchronous callback wrapper.

The DownloadService class in the gabrielbaute/spotify-saver repository provides the backbone for API-driven music downloads, enabling users to fetch tracks, albums, and playlists simultaneously without blocking the event loop. Understanding how this service orchestrates concurrent downloads with progress callbacks reveals a robust pattern for integrating synchronous I/O-heavy libraries like yt-dlp into modern async web frameworks such as FastAPI.

Async Architecture and Thread-Pool Concurrency

Non-blocking Public Interface

The service exposes four public coroutine methods in spotifysaver/api/services/download_service.py (lines 50-71): download_from_url, _download_track, _download_album, and _download_playlist. Declared with async def, these methods allow FastAPI request handlers to initiate downloads without blocking the event loop. Each method can be awaited independently, enabling true parallelism when multiple clients request downloads simultaneously.

Thread-Pool Offloading for Blocking Operations

Since yt-dlp performs blocking network I/O and file system operations, DownloadService dispatches these calls to a thread pool using loop.run_in_executor. This pattern prevents the async event loop from stalling during heavy downloads.

• Single tracks: await loop.run_in_executor(None, self._download_track_sync, track) (lines 90-94).
• Albums: await loop.run_in_executor(None, self.downloader.download_album_cli, ...) (lines 17-29).
• Playlists: await loop.run_in_executor(None, self.downloader.download_playlist_cli, ...) (lines 53-57).

Each coroutine receives its own executor task, allowing CPU-bound and I/O-bound work to proceed simultaneously across multiple threads while the main loop remains responsive.

Progress Callback Propagation

Callback Signature and Initialization

The API accepts an optional progress_callback parameter typed as Callable[[int, int, str], None], receiving the current index, total count, and track name. For single track downloads, the service invokes the callback immediately before heavy work begins at download_service.py lines 86-88:

progress_callback(1, 1, track.name)

Wrapper Functions for Collection Downloads

When downloading albums or playlists, the service bridges the async/sync boundary by creating a thin sync_progress_callback wrapper. This synchronous function forwards (idx, total, name) tuples to the original callback without blocking the thread pool worker.

• Album wrapper: Defined at lines 112-115 in download_service.py, passed to download_album_cli.
• Playlist wrapper: Defined at lines 149-152 in download_service.py, passed to download_playlist_cli.

CLI Downloader Integration

The YouTubeDownloaderForCLI class in spotifysaver/downloader/youtube_downloader_for_cli.py respects the same callback contract:

• Single tracks: Callback fired once before yt-dlp invocation (lines 66-68).
• Albums: Callback invoked inside the track loop with progress_callback(idx, len(album.tracks), track.name) (lines 24-27).
• Playlists: Analogous per-track updates within the collection loop (lines 90-92).

Practical Implementation Example

The following example demonstrates how to initiate multiple concurrent downloads while receiving real-time progress updates:

import asyncio
from spotifysaver.api.services.download_service import DownloadService

async def main():
    service = DownloadService(output_dir="MyMusic", output_format="mp3")
    
    # Simple progress printer

    def show_progress(current, total, name):
        print(f"[{current}/{total}] {name}")

    # Start three downloads concurrently

    urls = [
        "https://open.spotify.com/track/abc123",
        "https://open.spotify.com/album/def456",
        "https://open.spotify.com/playlist/ghi789",
    ]

    tasks = [
        service.download_from_url(url, progress_callback=show_progress)
        for url in urls
    ]

    results = await asyncio.gather(*tasks, return_exceptions=True)
    for res in results:
        print("Result:", res)

asyncio.run(main())

Under the hood, download_from_url determines the URL type and launches the appropriate internal coroutine. Each coroutine hands the heavy lifting to run_in_executor, enabling the three downloads to progress in parallel while show_progress receives callbacks for every track.

Result Aggregation and Thread Safety

Each coroutine returns a dictionary summarizing successes, failures, total_tracks, and the output_dir. Because run_in_executor isolates blocking work in separate threads while the async orchestrator manages flow control, the API can safely collect results from many concurrent downloads without race conditions.

The progress callback mechanism remains thread-safe because the wrapper functions execute synchronously within the thread context, merely passing data back to the caller's callback mechanism without shared state modifications.

Summary

• DownloadService exposes an async interface (async def) that allows FastAPI to handle multiple download requests concurrently without blocking.
• Thread-pool offloading via loop.run_in_executor executes blocking yt-dlp operations in parallel while keeping the event loop responsive.
• A synchronous wrapper pattern (sync_progress_callback) bridges the gap between thread-pool workers and the async callback interface, enabling real-time progress updates for albums and playlists.
• The implementation spans two key files: spotifysaver/api/services/download_service.py for orchestration and spotifysaver/downloader/youtube_downloader_for_cli.py for the actual download logic.

Frequently Asked Questions

How does the service prevent blocking the FastAPI event loop during downloads?

The service uses asyncio.get_event_loop().run_in_executor() to dispatch all blocking yt-dlp operations to a background thread pool. This keeps the main async event loop free to handle incoming HTTP requests while downloads proceed in parallel threads, as implemented in download_service.py lines 90-94 for individual tracks.

What is the exact signature for the progress callback function?

The callback must accept three parameters: current (int), total (int), and name (str), making the full type signature Callable[[int, int, str], None]. This contract applies uniformly to single tracks, albums, and playlist downloads throughout the codebase.

Can multiple users download different playlists simultaneously without interfering with each other's progress updates?

Yes. Each invocation of download_from_url creates an independent coroutine with its own callback instance. Because the callbacks are passed as arguments to thread-pool tasks in run_in_executor, they remain isolated per request, preventing cross-contamination of progress state between concurrent users.

Where does the actual file writing and yt-dlp processing occur?

The heavy lifting happens in spotifysaver/downloader/youtube_downloader_for_cli.py, specifically within methods like download_track_cli, download_album_cli, and download_playlist_cli. These synchronous methods are invoked by DownloadService through the thread pool executor, with progress callbacks fired at lines 66-68, 24-27, and 90-92 respectively.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}