How the VPhoneScreenRecorder Capture Pipeline Works: From Virtual Display to MOV File

The VPhoneScreenRecorder capture pipeline extracts raw pixel data from a private VZGraphicsDisplay object using Objective-C runtime calls, encodes frames via AVAssetWriter at 30 FPS, and outputs a QuickTime MOV file to the Desktop.

The capture pipeline in the Lakr233/vphone-cli repository enables screen recording of virtualized iOS devices by bridging Apple's private Virtualization framework APIs with Swift-based media encoding. This article examines the internal machinery of VPhoneScreenRecorder, tracing how it captures video frames without public APIs and writes them to disk.

Architecture of the VPhoneScreenRecorder Capture Pipeline

The VPhoneScreenRecorder capture pipeline consists of four distinct stages orchestrated within sources/vphone-cli/VPhoneScreenRecorder.swift. Each stage transforms the data from raw display output to serialized video frames.

Stage 1: Source Resolution via resolveCaptureSource

When recording initiates, the pipeline first resolves a private graphics display object. The method resolveCaptureSource(for:) (lines 51‑56) down-casts the NSView parameter to a VPhoneVirtualMachineView and extracts its recordingGraphicsDisplay property.

This returns a CaptureSource struct containing the VZGraphicsDisplay instance and a textual description for logging. The VZGraphicsDisplay represents the virtual machine's frame buffer and serves as the raw pixel source for the entire capture pipeline.

Stage 2: Frame Acquisition Loop

The startRecording(view:) method (lines 52‑96) configures the encoding infrastructure. It initializes an AVAssetWriter and an AVAssetWriterInputPixelBufferAdaptor to manage pixel buffer pooling.

A Timer fires at 30 Hz (line 96), driving the captureFrame() method on each tick. This method verifies that the AVAssetWriterInput is ready for media data, then forwards the display object to captureGraphicsDisplayFrame(_:adaptor:) for screenshot extraction.

Stage 3: Screenshot Extraction via Objective-C Runtime

The critical extraction logic resides in captureGraphicsDisplayFrame(_:adaptor:) (lines 84‑92). This method guards against concurrent captures, then calls takeGraphicsScreenshot(from:completion:) (lines 68‑85).

Because Apple does not expose screenshot functionality publicly, the code uses runtime introspection to invoke _takeScreenshotWithCompletionHandler:. It obtains the method implementation via class_getInstanceMethod and method_getImplementation, then invokes it with a Swift completion block that receives an opaque image object. The helper convertScreenshotObject(_:) (lines 95‑108) casts this object to a CGImage suitable for encoding.

Stage 4: Encoding and Writing to Disk

The final stage occurs in appendFrame(from:cgImage:) (lines 21‑48). This method retrieves a pixel buffer from the adaptor's pool, draws the CGImage into the buffer using CGContext, and timestamps the frame using CMTime(value: frameCount, timescale: 30) (line 46).

The encoded frame appends to the writer's input (line 47). When stopRecording() executes, the timer invalidates, the writer finalizes, and the resulting .mov file persists to the user's Desktop (lines 8‑30).

Implementation Details in VPhoneScreenRecorder.swift

The VPhoneScreenRecorder.swift file implements the capture pipeline without relying on public Virtualization framework APIs for screen capture.

Accessing Private Virtualization APIs

The pipeline relies entirely on the private selector _takeScreenshotWithCompletionHandler: available on VZGraphicsDisplay objects. The recorder dynamically looks up this selector using sel_registerName and executes it via method_getImplementation, bypassing compile-time visibility checks.

This technique allows vphone-cli to capture frames from the virtual iOS device where standard macOS screen recording APIs would fail or capture the host window chrome instead.

Timer-Driven Frame Capture

Unlike display link callbacks, the capture pipeline uses an Timer scheduled at a fixed 30 Hz interval. This design choice ensures consistent frame timing for the AVAssetWriter while avoiding dependencies on the virtual display's refresh rate.

The timer runs on the main run loop, calling captureFrame() which synchronously extracts and encodes each frame before the next timer tick.

Taking Still Screenshots vs. Video Recording

The VPhoneScreenRecorder class reuses the capture pipeline infrastructure for single-frame captures through captureStillImage(from:) (lines 13‑18). This method leverages the same source resolution and Objective-C runtime screenshot logic, but instead of feeding frames to AVAssetWriter, it writes the resulting CGImage directly to disk via saveScreenshot or copies it to the system pasteboard via copyScreenshotToPasteboard.

Still captures skip the timer infrastructure and pixel buffer pooling, making them lighter for one-off operations while maintaining identical image quality to video frames.

Code Examples for the Capture Pipeline

Start a recording session from a menu command or button handler:

let recorder = VPhoneScreenRecorder()
try recorder.startRecording(view: vmView)   // vmView is the VPhoneVirtualMachineView

Stop the recording and retrieve the saved file path:

if let url = await recorder.stopRecording() {
    print("Recording saved at \(url.path)")
}

Capture a single screenshot and copy it to the clipboard:

try await recorder.copyScreenshotToPasteboard(view: vmView)

Save a still screenshot as a file:

let fileURL = try await recorder.saveScreenshot(view: vmView)
print("Screenshot saved at \(fileURL.path)")

Summary

• The capture pipeline resolves a private VZGraphicsDisplay from VPhoneVirtualMachineView via resolveCaptureSource(for:) in VPhoneScreenRecorder.swift.
• A 30 Hz timer drives the frame loop in startRecording(view:), invoking captureFrame() to extract screenshots.
• Runtime introspection calls _takeScreenshotWithCompletionHandler: to obtain raw pixel data without public APIs.
• appendFrame(from:cgImage:) encodes frames into an AVAssetWriter outputting a Desktop MOV file.
• Still images reuse the extraction logic but bypass the timer and video encoding stages.

Frequently Asked Questions

How does VPhoneScreenRecorder access the virtual machine's display without public APIs?

The recorder accesses the display through the private VZGraphicsDisplay class provided by Apple's Virtualization framework. It retrieves this object from VPhoneVirtualMachineView.swift via the recordingGraphicsDisplay property, then uses Objective-C runtime functions to invoke the undocumented _takeScreenshotWithCompletionHandler: selector.

What frame rate does the VPhoneScreenRecorder capture pipeline use?

The capture pipeline samples the virtual display at a fixed 30 frames per second. This is implemented via a Timer scheduled in startRecording(view:) at line 96, which triggers frame capture regardless of the virtual device's actual refresh rate.

Where does VPhoneScreenRecorder save recorded video files?

When stopRecording() completes, the finalized QuickTime movie saves to the user's Desktop directory. The file uses the .mov extension and contains H.264-encoded video with timestamps derived from a 30-timescale CMTime counter.

Can I use the capture pipeline for automated screenshot testing?

Yes. The captureStillImage(from:) method exposes the same screenshot extraction logic used during video recording, making it suitable for automation. You can call saveScreenshot(view:) or copyScreenshotToPasteboard(view:) from test harnesses to capture the current virtual machine state without starting the timer-driven video encoding process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}