How youtube-dl Handles Retries and Error Recovery During Downloads

youtube-dl implements a hierarchical retry system where global retries settings control HTTP connection attempts while separate fragment_retries manage DASH/HLS segment downloads, using specific exception handling in HttpFD and FragmentFD classes to distinguish between transient network errors and fatal failures.

The ytdl-org/youtube-dl repository provides a robust download engine designed to handle unstable network conditions through configurable retry and error recovery mechanisms. Understanding how youtube-dl retries and error recovery works is essential for building resilient media pipelines and troubleshooting failed downloads.

Configuring youtube-dl Retries: Global Settings

youtube-dl exposes retry behavior through two primary configuration layers: global download retries and fragment-specific retries.

Command-Line and API Configuration

The user controls the number of retry attempts via the --retries command-line option or the retries key in the Python API. This value is stored in self.params['retries'] and consulted by all downloader implementations. The default value is 0, meaning no automatic retry occurs unless explicitly configured.

# Retry up to 10 times on HTTP 5xx or connection errors

youtube-dl --retries 10 https://example.com/video.mp4

from youtube_dl import YoutubeDL

ydl_opts = {
    'retries': 5,                 # global HTTP/HTTPS retries

    'fragment_retries': 3,        # retries per DASH/HLS fragment

    'continuedl': True,           # enable resume support

}
with YoutubeDL(ydl_opts) as ydl:
    ydl.download(['https://example.com/video'])

HTTP/HTTPS Download Retry Logic in HttpFD

The HttpFD class in youtube_dl/downloader/http.py implements the core retry mechanism for standard HTTP downloads.

The Core Retry Loop

The download method wraps the connection logic in a while loop that increments a counter until it exceeds the configured retry limit:

# From youtube_dl/downloader/http.py lines 47-53

count = 0
while count <= retries:
    try:
        # Establish connection and download

        establish_connection()
        # ... download logic ...

    except RetryDownload as e:
        count += 1
        continue

Handling Transient Network Errors

When HttpFD encounters specific transient errors, it raises a custom RetryDownload exception rather than failing immediately. The loop catches this exception, increments the attempt counter, reports the retry via self.report_retry(), and re-establishes the connection.

Transient errors that trigger retries include:

• Socket timeouts (socket.timeout)
• Connection resets (socket.error)
• HTTP 5xx server errors
• Short-read errors (incomplete data transfer)

The reporting mechanism formats user-visible messages in youtube_dl/downloader/common.py (lines 23-28):

self.to_screen('[download] Got server HTTP error: %s. Retrying (attempt %d of %s)...' %
               (error_to_compat_str(err), count, self.format_retries(retries)))

HTTP 416 Range Errors and Resume Failures

When resuming partial downloads, HttpFD handles HTTP 416 Range Not Satisfiable errors intelligently. If the server returns a 416 status code, the downloader examines the actual file size. If the file is already fully downloaded, it treats the download as complete rather than failing. If resume attempts fail due to missing or mismatching Content-Range headers, the downloader discards the partial file, reports "Unable to resume", and starts a fresh download (see http.py lines 41-48 and 51-69).

Fragment-Level Retries for DASH and HLS Streams

Adaptive streaming protocols like DASH and HLS download content in small segments (fragments), requiring specialized retry logic distinct from single-file HTTP downloads.

FragmentFD and fragment_retries

Fragment-based downloaders inherit from FragmentFD in youtube_dl/downloader/fragment.py. They use the fragment_retries parameter (default 0) to control per-fragment attempts. Each fragment download occurs inside an infinite loop that breaks only on success or when exhausting the retry limit:

# From youtube_dl/downloader/dash.py lines 51-67

for count in itertools.count():
    try:
        # Download fragment

        frag_content = download_fragment()
        break  # Success

    except compat_urllib_error.HTTPError as err:
        self.report_retry_fragment(err, frag_index, count + 1, fragment_retries)
        if count >= fragment_retries:
            raise
        continue

Protecting MP4 Header Integrity

The fragment downloader implements special handling for the first fragment, which typically contains MP4 initialization data (header). Non-HTTP errors occurring during the first fragment download (such as DownloadError) cause an immediate abort rather than a retry, preserving file integrity. Subsequent fragments may retry on broader error types (see dash.py lines 68-72).

User Feedback and Reporting Mechanisms

youtube-dl provides consistent user feedback through standardized reporting methods defined in the base downloader classes.

The FileDownloader.report_retry method in youtube_dl/downloader/common.py (lines 23-28) formats retry notifications:

self.to_screen('[download] Got server HTTP error: %s. Retrying (attempt %d of %s)...' %
               (error_to_compat_str(err), count, self.format_retries(retries)))

Fragment-specific retries use report_retry_fragment defined in FragmentFD (youtube_dl/downloader/fragment.py), following a similar pattern but including fragment indices for context.

When all retry attempts are exhausted, the downloader calls self.report_error('giving up after %s retries' % retries) (see http.py lines 61-63), providing clear failure indication to the user.

Summary

youtube-dl implements a sophisticated, multi-layered retry architecture that distinguishes between transient network failures and permanent errors:

• Global retry control via the retries parameter (default 0) stored in self.params['retries'] and checked by all downloaders
• HTTP-specific recovery in HttpFD using RetryDownload exceptions to handle timeouts, connection resets, and HTTP 5xx errors with intelligent 416 range handling
• Fragment-level resilience through FragmentFD and fragment_retries, protecting MP4 header integrity by aborting immediately on non-HTTP first-fragment errors
• User transparency via standardized report_retry and report_error methods in common.py and fragment.py

Frequently Asked Questions

How do I set the number of retries in youtube-dl?

Use the --retries command-line option followed by the desired number of attempts, or set the retries key in the Python API options dictionary. The default value is 0, meaning youtube-dl will not automatically retry failed downloads unless explicitly configured. For fragment-based streams like DASH or HLS, use the separate fragment_retries option.

What types of errors trigger a retry in youtube-dl?

youtube-dl retries transient network errors including socket timeouts, connection resets, and HTTP 5xx server errors. In the HttpFD class, these conditions raise a RetryDownload exception that triggers the retry loop. However, HTTP 4xx client errors (except 416 range errors) and file integrity failures typically do not trigger retries. For DASH/HLS downloads, only HTTP errors trigger fragment retries; other DownloadError types abort immediately, especially for the first fragment containing MP4 headers.

Does youtube-dl retry individual video fragments separately?

Yes. Fragment-based downloaders inheriting from FragmentFD use the fragment_retries parameter to retry individual segments independently. This is implemented in youtube_dl/downloader/dash.py using an itertools.count() loop that catches compat_urllib_error.HTTPError and reports retries via report_retry_fragment. If a fragment fails after exhausting fragment_retries, the entire download aborts.

What happens when youtube-dl exhausts all retry attempts?

When the retry counter exceeds the configured limit (either retries for HTTP or fragment_retries for fragments), youtube-dl calls self.report_error() with a message like "giving up after N retries" (see http.py lines 61-63). The download fails and youtube-dl proceeds to the next item in the queue or exits if processing a single URL. For resume-capable downloads, youtube-dl may discard partial files and attempt a fresh download if range requests fail, rather than retrying the same failed connection indefinitely.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}