# How youtube-dl Implements Format Selection with Selector Expressions

> Discover how youtube-dl uses format selector expressions to filter merge and prioritize stream dictionaries at runtime. Learn the technical details behind this powerful feature.

- Repository: [youtube-dl/youtube-dl](https://github.com/ytdl-org/youtube-dl)
- Tags: internals
- Published: 2026-02-25

---

**youtube-dl compiles textual format selector expressions into executable Python functions that filter, merge, and prioritize stream dictionaries at runtime.**

youtube-dl enables precise media downloads through **format selection with selector expressions** passed via the `--format` flag. The implementation transforms compact strings—such as `bestvideo+bestaudio/best`—into executable selector functions that operate on the raw format metadata extracted from video hosts. This article examines the complete pipeline, from tokenization to execution, based on the source code in [`youtube_dl/YoutubeDL.py`](https://github.com/ytdl-org/youtube-dl/blob/main/youtube_dl/YoutubeDL.py).

## Overview of the Format Selector Pipeline

The format selection system processes user input through six distinct stages before yielding downloadable formats.

1. **Tokenization** – The input string is split into tokens using Python’s `tokenize` module.
2. **Cleaning** – The `_remove_unused_ops` function strips irrelevant operators and merges adjacent name tokens (e.g., `"mp4" "-" "baseline"` becomes `"mp4-baseline"`).
3. **Parsing** – A recursive descent parser (`_parse_format_selection`) constructs a tree of `FormatSelector` objects representing the grammar.
4. **Compilation** – The tree is transformed into a selector function via `_build_selector_function`.
5. **Execution** – The compiled function receives a **context** (`ctx`) containing the list of formats and yields the final selection.

This flow is orchestrated in `YoutubeDL.extract_info` (lines 1828–1856):

```python
format_selector = self.build_format_selector(req_format)
formats_to_download = list(format_selector(ctx))

```

*Source:* [`YoutubeDL.py#L1828-L1856`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1828-L1856)

## Selector Expression Grammar and Token Handling

The parser recognizes four fundamental selector types defined as constants at lines 1319–1322:

| Constant | Meaning |
|----------|---------|
| `PICKFIRST` | `A / B` – pick the first branch that yields any format. |
| `MERGE` | `A + B` – combine a video-only format with an audio-only format. |
| `SINGLE` | A plain token like `best` or a specific `format_id`. |
| `GROUP` | Parenthesized sub-expressions for precedence. |

*Source:* [`YoutubeDL.py#L1319-L1322`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1319-L1322)

### Cleaning the Token Stream

Operators that have no semantic meaning for the selector (`-`, `:`, etc.) are discarded by `_remove_unused_ops` (lines 1334–1362). Only `/`, `+`, `,`, `(`, and `)` are retained. Consecutive name tokens are merged into single identifiers, ensuring that format specifications like `mp4-baseline` are treated as atomic units rather than separate tokens.

*Source:* [`YoutubeDL.py#L1334-L1362`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1334-L1362)

### Recursive Descent Parsing

`_parse_format_selection` walks the token list recursively, creating `FormatSelector` nodes according to the operators encountered:

- **`,`** – Separates alternative selectors within a merge or choice.
- **`/`** – Creates a `PICKFIRST` node representing fallback selection.
- **`+`** – Creates a `MERGE` node for combining video and audio streams.
- **`[`** – Starts a filter block; enclosed text is stored in `selector.filters`.
- **`(`** – Creates a `GROUP` node for nested expressions.

*Source:* [`YoutubeDL.py#L1364-L1395`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1364-L1395)

## Compiling Format Selectors into Executable Functions

`_build_selector_function` transforms the `FormatSelector` tree into a Python generator function that operates on a context dictionary (`ctx`). This compilation step (lines 1400–1550) handles each selector type with specific logic:

- **GROUP** – Delegates evaluation to the nested sub-selector.
- **PICKFIRST** – Evaluates branches sequentially and returns the first non-empty result.
- **SINGLE** – Handles built-in shortcuts (`best`, `worst`, `bestaudio`, `bestvideo`) or matches by `format_id` and `ext`. The implementation distinguishes three matching families:
  1. *Special keywords* – Select highest/lowest quality after filtering by stream type.
  2. *Extension shortcuts* – Match file extensions (e.g., `mp4` selects the last matching format).
  3. *Exact format ID* – Direct string match against `format_id`.
- **MERGE** – Retrieves separate video and audio format dictionaries, then returns a merged dictionary where `format_id` becomes `"video_id+audio_id"` and other fields are combined.

*Source (single selector):* [`YoutubeDL.py#L1442-L1490`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1442-L1490)  
*Source (merge handling):* [`YoutubeDL.py#L1495-L1512`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1495-L1512)

## Context Construction and Runtime Execution

Before invoking the selector, `YoutubeDL.extract_info` builds a **context** (`ctx`) that provides the runtime environment for format selection. The context is constructed at lines 1845–1849:

```python
ctx = {
    'formats': formats,
    'incomplete_formats': all_video_only or all_audio_only,
}

```

The `incomplete_formats` boolean indicates whether the extractor provided exclusively video-only or audio-only streams. This flag influences the fallback logic for keywords like `best` and `worst`, ensuring the selector adapts to incomplete metadata rather than failing to match.

*Source:* [`YoutubeDL.py#L1845-L1849`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1845-L1849)

## Practical Examples of Format Selection

### Command-Line Usage

The most common way to invoke **format selection with selector expressions** is via the `-f` or `--format` flag:

```bash

# Download best video merged with best audio, fallback to best combined

youtube-dl -f 'bestvideo+bestaudio/best' 'https://www.youtube.com/watch?v=abcd1234'

# Select specific height and extension with filters

youtube-dl -f 'bestvideo[height<=720][ext=mp4]+bestaudio[ext=m4a]/best' URL

```

### Programmatic Usage

Developers can build and test selectors programmatically without downloading:

```python
from youtube_dl import YoutubeDL

# Initialize with a format expression

ydl_opts = {'format': 'bestvideo[height<=720]+bestaudio/best'}
ydl = YoutubeDL(ydl_opts)

# Extract info without downloading to get available formats

info = ydl.extract_info('https://www.youtube.com/watch?v=abcd1234', download=False)

# Build the selector function manually

selector = ydl.build_format_selector(ydl.params['format'])

# Create a mock context to inspect selection logic

ctx = {
    'formats': info['formats'],
    'incomplete_formats': False
}

# Execute and inspect chosen formats

chosen = list(selector(ctx))
print('Chosen format IDs:', [f['format_id'] for f in chosen])

```

This approach allows inspection of which formats would be selected before initiating the actual download.

## Key Source Files

| File | Role |
|------|------|
| [`youtube_dl/YoutubeDL.py`](https://github.com/ytdl-org/youtube-dl/blob/main/youtube_dl/YoutubeDL.py) | Implements `build_format_selector`, the parser, compiler, and runtime context handling. |
| [`youtube_dl/options.py`](https://github.com/ytdl-org/youtube-dl/blob/main/youtube_dl/options.py) | Defines the `--format` CLI option and documentation. |
| `youtube_dl/extractor/*` | Supplies the raw `formats` list that selectors operate on. |

*Direct links:*

- Core selector implementation – [`YoutubeDL.py#L1312-L1550`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1312-L1550)
- Context creation – [`YoutubeDL.py#L1845-L1854`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L1845-L1854)
- Option definition – [[`options.py`](https://github.com/ytdl-org/youtube-dl/blob/main/options.py)](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/options.py) (search for `'format'`)

## Summary

- **youtube-dl** implements **format selection with selector expressions** by compiling textual specifications into executable Python functions.
- The pipeline involves six stages: tokenization, cleaning (`_remove_unused_ops`), parsing (`_parse_format_selection`), compilation (`_build_selector_function`), and execution against a context.
- The grammar supports four selector types: `PICKFIRST` (fallback `/`), `MERGE` (combine `+`), `SINGLE` (specific formats), and `GROUP` (nested expressions).
- The context dictionary (`ctx`) supplies format metadata and an `incomplete_formats` flag to handle edge cases where extractors return only video or audio streams.
- Developers can programmatically build and test selectors using `build_format_selector` without initiating downloads.

## Frequently Asked Questions

### What is the syntax for combining video and audio formats in youtube-dl?

Use the plus operator (`+`) to merge a video-only format with an audio-only format. For example, `bestvideo+bestaudio` selects the highest quality video stream and the highest quality audio stream, then combines them into a single download. If you need a fallback when separate streams are unavailable, append `/best` to use the best combined format instead.

### How does youtube-dl handle format selection when only video or only audio is available?

The selector receives an `incomplete_formats` flag in the context dictionary when the extractor provides exclusively video-only or audio-only streams. This boolean influences the fallback logic for keywords like `best` and `worst`, ensuring the selector adapts to incomplete metadata rather than failing to match any formats.

### Can I test format selectors programmatically without downloading the video?

Yes. Instantiate `YoutubeDL` with your desired format string, call `extract_info` with `download=False` to retrieve format metadata, then invoke `build_format_selector` to create the selector function. Construct a mock context dictionary with the `formats` list and `incomplete_formats` flag, and call the selector with this context to inspect which format IDs would be chosen.

### What happens if my format selector expression contains invalid syntax?

The parser will raise a `SyntaxError` during the tokenization or parsing phase (`_parse_format_selection`). Specifically, if the recursive descent parser encounters unexpected tokens, unbalanced parentheses, or malformed filter expressions, it raises `SyntaxError` with a descriptive message indicating the invalid component of your format expression.