How to Use and Customize fake_headers in you-get for Site-Specific Requests

The fake_headers dictionary in you-get simulates a real browser's HTTP headers to bypass bot detection, and you can customize it by copying the base dictionary from common.py and modifying specific values like User-Agent or Referer before passing it to download functions.

The you-get open-source downloader relies on fake_headers to masquerade as a legitimate web browser when fetching media from video hosting sites. Defined in src/you_get/common.py at line 143, this global header set prevents servers from rejecting requests that lack typical browser fingerprints such as a realistic User-Agent or proper Accept encoding declarations.

What Is fake_headers and Why It Matters

fake_headers is a Python dictionary containing standard HTTP headers that mimic a modern desktop browser. According to the you-get source code, the default set includes:

• Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
• Accept-Charset: UTF-8,*;q=0.5
• Accept-Encoding: gzip,deflate,sdch
• Accept-Language: en-US,en;q=0.8
• User-Agent: A current Edge browser string (e.g., Mozilla/5.0 (Windows NT 10.0; Win64; x64)... Edge/126.0...)

These headers serve three critical purposes:

1. Bypassing bot filters – Many video platforms return 403 Forbidden errors to requests missing a realistic User-Agent.
2. Avoiding CDN redirects – Content delivery networks may serve different media qualities based on Accept-Language or encoding headers.
3. Handling site-specific quirks – Some extractors require mobile headers or authentication cookies to access restricted content.

How fake_headers Works in the you-get Architecture

The you-get library injects fake_headers at multiple layers of the HTTP stack. In src/you_get/common.py, the dictionary is defined globally and then referenced by utility functions and site extractors.

When you call get_html(url, faker=True), the function automatically uses fake_headers['User-Agent'] for the request. Similarly, extractors like those in src/you_get/extractors/vimeo.py (line 50) and src/you_get/extractors/zhihu.py (line 27) explicitly pass headers=fake_headers to get_content to ensure the request appears to come from a standard browser.

Some extractors extend this pattern further. In src/you_get/extractors/showroom.py (line 13), the code creates a completely separate fake_headers_mobile dictionary optimized for mobile endpoints, demonstrating how the base pattern can be forked for device-specific requirements.

Customizing fake_headers for Specific Sites

To adapt fake_headers for a particular video host, always copy the base dictionary before modifying it. This prevents accidental mutation of the global state used by other extractors.

Changing the User-Agent for Mobile Sites

Some platforms serve different video formats to mobile browsers. To impersonate an iPhone user:

from you_get.common import fake_headers
from you_get.util import get_content

# Copy the default headers

mobile_headers = fake_headers.copy()

# Replace with iPhone Safari User-Agent

mobile_headers['User-Agent'] = (
    'Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) '
    'AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 '
    'Safari/604.1'
)

# Fetch mobile-specific content

html = get_content('https://example.com/video/123', headers=mobile_headers)

Adding Referer Headers for Hotlink Protection

Sites like Vimeo or Bilibili often verify the Referer header to prevent direct linking. Add this to your custom set:

my_headers = fake_headers.copy()
my_headers['Referer'] = 'https://example.com/gallery'
html = get_content('https://example.com/video/456', headers=my_headers)

Including Cookies for Authentication

For premium content requiring login, inject session cookies into the header dictionary:

auth_headers = fake_headers.copy()
auth_headers['Cookie'] = 'sessionid=abcd1234; path=/; Secure; HttpOnly'
html = get_content('https://private.example.com/video/789', headers=auth_headers)

Creating Site-Specific Header Sets

For extractors that require fundamentally different fingerprints, define a new dictionary inspired by the original. The showroom.py extractor demonstrates this pattern by creating fake_headers_mobile for Android devices rather than modifying the global desktop set:

# Pattern from src/you_get/extractors/showroom.py

fake_headers_mobile = {
    'User-Agent': ('Mozilla/5.0 (Linux; Android 13; Pixel 7 Pro) '
                   'AppleWebKit/537.36 (KHTML, like Gecko) '
                   'Chrome/126.0.0.0 Mobile Safari/537.36')
}
html = get_content(webpage_url, headers=fake_headers_mobile)

Summary

• fake_headers in src/you_get/common.py provides a global dictionary of browser-like HTTP headers to prevent bot detection.
• Default coverage includes standard Accept encodings, language preferences, and a current Edge browser User-Agent.
• Integration occurs through the faker=True parameter in get_html() or explicit headers=fake_headers arguments in extractors like vimeo.py and zhihu.py.
• Customization requires copying the base dictionary (e.g., fake_headers.copy()) before adding site-specific values like mobile User-Agent strings, Referer headers, or authentication cookies.
• Mobile variants can be created from scratch, as demonstrated in showroom.py with fake_headers_mobile.

Frequently Asked Questions

Where is fake_headers defined in you-get?

The fake_headers dictionary is defined in src/you_get/common.py at line 143. This file serves as the central configuration point for HTTP headers that mimic a modern desktop browser, including the User-Agent, Accept encodings, and language preferences.

How do I use fake_headers without modifying it?

To use the default headers without customization, pass faker=True to utility functions like get_html() or explicitly pass headers=fake_headers to functions like get_content(). For example: html = get_html('https://example.com', faker=True). This approach uses the global dictionary directly without creating a local copy.

Can I use fake_headers for sites requiring login?

Yes, but you must customize the dictionary to include authentication tokens. Copy the base dictionary with custom_headers = fake_headers.copy(), then add your session cookie via custom_headers['Cookie'] = 'sessionid=your_token'. Pass this customized dictionary to get_content() or download_urls() to access premium content while maintaining the browser-like fingerprint for other headers.

What is the difference between faker=True and headers=fake_headers?

The faker=True parameter is a convenience flag used in high-level utilities like get_html() that automatically injects only the User-Agent from fake_headers. In contrast, passing headers=fake_headers explicitly provides the complete dictionary (including Accept, Accept-Language, etc.) to low-level request functions. Extractors in vimeo.py and zhihu.py prefer the explicit approach for full control over the request fingerprint.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}