How to Implement Portrait/Vertical Video Detection and Scaling in the Video-Use Extraction Pipeline

The Video-Use pipeline automatically detects vertical orientation using ffprobe and applies aspect-ratio-preserving scaling via FFmpeg's scale filters.

The browser-use/video-use repository handles mixed-orientation source footage by analyzing each video's dimensions before extraction. This ensures that portrait videos from mobile devices retain their aspect ratio while fitting within standard 1080p or 720p output constraints.

How Portrait Detection Works

The detection logic lives in helpers/render.py within the is_portrait_source function. This utility queries the video metadata using ffprobe and compares the height and width values.

def is_portrait_source(video: Path) -> bool:
    """Return True if the video's height > width (portrait / vertical)."""
    try:
        out = subprocess.run(
            ["ffprobe", "-v", "error", "-select_streams", "v:0",
             "-show_entries", "stream=width,height",
             "-of", "csv=p=0", str(video)],
            capture_output=True, text=True, check=True,
        )
        w, h = map(int, out.stdout.strip().split(","))
        return h > w
    except Exception:
        return False

• ffprobe reads the first video stream's dimensions via -show_entries stream=width,height.
• The function returns True when height exceeds width, indicating a vertical orientation.
• This check runs for every source video immediately before segment extraction begins.

Scaling Logic for Vertical Videos

Within extract_segment, the pipeline selects the appropriate FFmpeg scale filter based on the portrait flag. The -2 value preserves the original aspect ratio by automatically calculating the missing dimension.

portrait = is_portrait_source(source)
if draft:
    scale = "scale=-2:1280" if portrait else "scale=1280:-2"
else:
    scale = "scale=-2:1920" if portrait else "scale=1920:-2"

• Portrait sources fix the height to the target resolution (1280px for draft mode, 1920px for final output) while the width scales proportionally.
• Landscape sources fix the width instead, allowing the height to adjust automatically.
• This logic ensures 1080p-height final outputs (or 720p for drafts) regardless of input orientation.

The selected scale string integrates into the video filter chain alongside optional HDR tonemapping and color grading:

vf_parts: list[str] = []
if is_hdr_source(source):
    vf_parts.append(TONEMAP_CHAIN)
vf_parts.append(scale)
if grade_filter:
    vf_parts.append(grade_filter)
vf = ",".join(vf_parts)

Pipeline Integration

The portrait detection operates at the per-segment extraction stage of the pipeline:

1. EDL parsing – The user provides an Edit Decision List (EDL) describing source clips and cut ranges.
2. extract_all_segments iterates over each range, resolves the source path, and calls extract_segment.
3. extract_segment runs is_portrait_source → selects the appropriate scale filter → builds the FFmpeg command that extracts, grades, fades, and writes the segment.
4. concat_segments joins the resulting per-segment MP4s into the final output.

All processing logic is contained in helpers/render.py, requiring no modifications to other modules to support vertical video handling.

Extending Portrait Detection

You can reuse is_portrait_source in other utilities, such as thumbnail generators or timeline views, by importing the function directly from the render helper. Because it relies only on ffprobe, it runs safely from any script context.

from helpers.render import is_portrait_source

if is_portrait_source(video_path):
    thumb_scale = "scale=-2:200"
else:
    thumb_scale = "scale=200:-2"

Code Examples

Extracting a Single Segment with Portrait-Aware Scaling

from pathlib import Path
from helpers.render import extract_segment, resolve_grade_filter

src = Path("samples/vertical_clip.mp4")
start, duration = 5.0, 12.3
grade = resolve_grade_filter("auto")     # or a raw ffmpeg filter string

out = Path("tmp/segment.mp4")

extract_segment(src, start, duration, grade, out, preview=False, draft=False)

The function automatically detects the vertical orientation and applies scale=-2:1920 to maintain the aspect ratio while fitting the 1080p height constraint.

Running the Full Pipeline on an EDL

python helpers/render.py edl.json -o final.mp4

The script parses the EDL, extracts each segment with the appropriate scaling, concatenates them, and produces final.mp4. No additional flags are required for portrait videos.

Summary

• Detection: The is_portrait_source function in helpers/render.py uses ffprobe to identify when height exceeds width.
• Scaling: Portrait videos use scale=-2:1920 (final) or scale=-2:1280 (draft) to fix height while preserving aspect ratio.
• Integration: The logic runs automatically within extract_segment during the per-segment extraction phase.
• Reusability: Import is_portrait_source directly for thumbnail generation or other orientation-dependent workflows.

Frequently Asked Questions

How does the pipeline detect portrait orientation?

The pipeline calls is_portrait_source in helpers/render.py, which executes ffprobe to read the video stream's width and height. When the height value exceeds the width, the function returns True, triggering the vertical scaling logic.

What FFmpeg scale values are used for vertical videos?

For portrait sources, the pipeline uses scale=-2:1920 for final output (1080p height) and scale=-2:1280 for draft mode (720p height). The -2 parameter tells FFmpeg to calculate the width automatically while maintaining the original aspect ratio.

Can I use the portrait detection function in other scripts?

Yes. Simply import is_portrait_source from helpers/render.py. The function is pure Python and only requires ffprobe to be available in the system path, making it safe to use in thumbnail generators, timeline views, or validation scripts.

Does this handle HDR content correctly?

Yes. The scaling filter integrates into the video filter chain after HDR tonemapping. The pipeline checks is_hdr_source and prepends the TONEMAP_CHAIN before appending the scale filter, ensuring HDR vertical videos are properly tone-mapped before scaling.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}