How ReClip Parses yt-dlp JSON Output: Handling Multi-Line stdout

ReClip parses yt-dlp JSON output by iterating over each line of stdout, returning the first valid JSON object while discarding empty lines to prevent "Extra data" decode errors when extractors emit multiple objects.

ReClip by averygan/reclip is a Flask-based application that extracts video metadata by invoking the yt-dlp command-line tool. When processing the -j flag output, the application addresses a critical edge case where certain extractors return multiple JSON objects sequentially, even with --no-playlist enabled. The implementation in app.py demonstrates a robust line-oriented parsing strategy that ensures reliable extraction of video information.

The Challenge with yt-dlp JSON Output

When ReClip retrieves video data using yt-dlp's -j option, the command outputs one JSON object per line to stdout. However, some extractors emit several JSON objects in sequence, causing a naive json.loads() call on the entire stdout string to fail with an "Extra data" error. Standard JSON decoders expect a single document per input string, making it necessary to handle stdout as a stream rather than a monolithic document.

The parse_ytdlp_json Implementation

The solution resides in the parse_ytdlp_json function defined in app.py at lines 16-29. This function accepts the raw stdout string, splits it into individual lines, and returns the first non-empty line that successfully deserializes as JSON.

def parse_ytdlp_json(stdout):
    """Parse yt-dlp JSON output.

    With ``-j`` yt-dlp prints one JSON object per line. Some extractors
    emit multiple videos even with ``--no-playlist``, so stdout contains
    several objects and a plain ``json.loads`` raises "Extra data".
    Return the first valid object.
    """
    for line in stdout.splitlines():
        line = line.strip()
        if not line:
            continue
        return json.loads(line)
    raise ValueError("yt-dlp returned no data")

This implementation guarantees that ReClip captures the primary video metadata regardless of trailing objects in the buffer. If the function encounters only empty lines or whitespace, it raises a ValueError with the specific message "yt-dlp returned no data", providing explicit error signaling for upstream exception handling.

Integration with the Flask API

The /api/info endpoint orchestrates the subprocess invocation and parsing workflow. When processing a request, ReClip constructs a command list restricting output to a single video, executes yt-dlp with a 60-second timeout, and passes the resulting stdout to the parser.

cmd = ["yt-dlp", "--no-playlist", "-j", url]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=60)
info = parse_ytdlp_json(result.stdout)

This pattern ensures that ReClip parses yt-dlp JSON output safely within the Flask request cycle, capturing video metadata without blocking on malformed or multi-object responses.

Practical Usage Examples

Fetching Video Info from the API

Client applications can retrieve parsed metadata by posting to the local endpoint:

import requests

url = "https://example.com/video"
resp = requests.post(
    "http://localhost:8899/api/info",
    json={"url": url}
)
data = resp.json()
print(data["title"])
print(data["thumbnail"])
print("Available formats:", data["formats"])

Directly Invoking the Parser for Debugging

Developers extending ReClip can test the parsing logic directly against yt-dlp output:

import subprocess, json
from app import parse_ytdlp_json

cmd = ["yt-dlp", "--no-playlist", "-j", "https://youtu.be/dQw4w9WgXcQ"]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=60)

# Parse the first JSON object from yt-dlp output

video_info = parse_ytdlp_json(result.stdout)

print(json.dumps(video_info, indent=2))

Summary

• ReClip handles multi-line JSON output from yt-dlp by processing stdout line-by-line rather than as a single document.
• The parse_ytdlp_json function in app.py (lines 16-29) returns the first valid JSON object and raises ValueError if no data is found.
• The implementation prevents "Extra data" JSON errors when extractors emit multiple objects despite the --no-playlist flag.
• Integration with the /api/info endpoint uses a 60-second timeout to ensure responsive error handling.

Frequently Asked Questions

Why does ReClip need a custom parser for yt-dlp JSON instead of using json.loads directly?

Standard json.loads() fails when the input string contains multiple JSON objects concatenated together. Because yt-dlp's -j flag can output several lines when certain extractors process complex URLs, parsing the entire stdout as a single JSON document triggers a decode error. ReClip's line-by-line approach isolates the first valid object while ignoring subsequent data.

What happens if yt-dlp returns an empty response or no valid JSON?

If parse_ytdlp_json encounters only empty lines or cannot find a valid JSON object after iterating through all stdout lines, it raises a ValueError with the message "yt-dlp returned no data". This allows the Flask API to catch the exception and return an appropriate error response to the client.

Does ReClip support parsing playlist data from yt-dlp?

The current implementation in app.py explicitly uses the --no-playlist flag when invoking yt-dlp, restricting output to a single video. While the parser itself could theoretically handle the first line of a playlist dump, the API endpoint is designed specifically for individual video metadata extraction as implemented in the averygan/reclip repository.

Where is the parse_ytdlp_json function located in the ReClip repository?

The function is defined in app.py at lines 16-29 within the averygan/reclip repository. This module contains the core backend logic including the Flask API routes and the yt-dlp integration layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}