How scrcpy Handles Audio Latency and Synchronization: A Deep Dive into the Source Code

scrcpy maintains low audio latency and tight synchronization through hardware timestamp extraction, a real-time audio regulator with FFmpeg compensation, and configurable buffering windows.

The Genymobile/scrcpy project streams Android audio to desktop clients with minimal delay, solving the classic problem of keeping audio synchronized with video frames during screen mirroring. Understanding how scrcpy handles audio latency and synchronization requires examining three complementary mechanisms implemented across the Java server and C client components.

Timestamp Extraction and Monotonic PTS Enforcement

At the capture stage, scrcpy extracts precise timing information from the Android audio subsystem to create presentation timestamps (PTS) that remain strictly monotonic.

Hardware Timestamp Retrieval

In server/src/main/java/com/genymobile/scrcpy/audio/AudioRecordReader.java, the read() method queries the AudioRecord instance for hardware timestamps:

int ret = recorder.getTimestamp(timestamp, AudioTimestamp.TIMEBASE_MONOTONIC);
if (ret == AudioRecord.SUCCESS && timestamp.nanoTime != previousRecorderTimestamp) {
    pts = timestamp.nanoTime / 1000;          // hardware PTS in µs
    previousRecorderTimestamp = timestamp.nanoTime;
}

The getTimestamp() call retrieves nanosecond-precision timing directly from the audio hardware, converting it to microseconds for the PTS value.

Fallback Estimation and Monotonicity Guarantees

When hardware timestamps are unavailable or repeat (indicating stale data), scrcpy falls back to software estimation while enforcing strict monotonicity:

} else {
    // fallback estimation when HW timestamp is unavailable
    pts = nextPts;
}
// Ensure monotonicity by adding minimal sample interval if needed
pts = Math.max(pts, previousPts + ONE_SAMPLE_US);

The ONE_SAMPLE_US constant represents the duration of a single audio sample, ensuring that even during estimation errors, the PTS never moves backward. This prevents audio glitches and maintains synchronization integrity.

Real-Time Audio Regulation and Clock Compensation

Once audio packets reach the desktop client, scrcpy employs a sophisticated regulator to manage buffering and correct clock drift between the Android device and the host computer.

Buffer Management and Target Levels

The audio regulator in app/src/audio_regulator.c maintains a target buffering level (default approximately 50ms) to balance latency against glitch resistance:

• Under-run protection: When the buffer falls below the target, the regulator inserts silence until sufficient samples accumulate, ensuring smooth playback startup.
• Overflow handling: When the buffer exceeds capacity, excess samples are dropped to prevent memory bloat.
• Discontinuity detection: The regulator detects PTS gaps larger than 100ms and triggers resynchronization, resetting the buffer state to realign with the incoming stream.

Dynamic Resampling with FFmpeg

To eliminate clock drift—the gradual desynchronization that occurs when the Android audio clock and host playback clock run at slightly different rates—the regulator uses FFmpeg's libswresample for dynamic compensation:

int64_t swr_delay = swr_get_delay(swr_ctx, ar->sample_rate);
int ret = swr_set_compensation(swr_ctx,
                               (int)(ar->target_buffering - current_buffering),
                               ar->sample_rate);

The swr_set_compensation() function adjusts the resampling ratio on-the-fly, effectively speeding up or slowing down the audio stream by tiny increments to match the video clock. This continuous correction happens without audible pitch shifts or interruptions, maintaining lip-sync accuracy over extended mirroring sessions.

Configurable Buffering for Latency Control

scrcpy exposes two command-line parameters that allow users to tune the trade-off between latency and stability:

• --audio-buffer: Sets the target buffered audio duration managed by the regulator (default ~50ms). Lower values reduce latency but increase vulnerability to glitches under network or CPU pressure.
• --audio-output-buffer: Configures the low-level playback buffer size (default ~5ms). This controls the OS audio FIFO depth, affecting how quickly the system responds to new samples.

Example configurations:

# Reduce latency to ~30ms (requires stable USB connection)

scrcpy --audio-buffer=30

# Increase robustness on congested networks

scrcpy --audio-buffer=100 --audio-output-buffer=20

These options are parsed in app/src/scrcpy.c and passed through to the audio regulator initialization, allowing real-time adjustment of the synchronization pipeline behavior.

The Complete Audio Pipeline Flow

Understanding how scrcpy handles audio latency and synchronization requires viewing the entire data flow:

1. Capture: On Android 11+, AudioRecordReader.java captures audio via AudioRecord, extracting hardware timestamps or estimating software timestamps while enforcing monotonicity with ONE_SAMPLE_US.

2. Encoding & Transport: Audio is encoded (typically Opus) and streamed over a local socket to the host client.

3. Decoding: The host decodes packets into raw PCM frames.

4. Regulation: sc_audio_regulator_push() processes frames, maintaining the target buffer level, inserting silence on under-run, and detecting discontinuities.

5. Compensation: The regulator calls swr_set_compensation() to adjust resampling ratios, correcting drift between the Android audio clock and host playback clock.

6. Playback: The regulated PCM stream passes to audio_player.c, which respects --audio-output-buffer settings and writes to the system sound card.

This architecture maintains approximately 50ms end-to-end latency by default while ensuring that audio remains locked to video frames through continuous clock compensation.

Summary

• Hardware timestamp extraction in AudioRecordReader.java provides microsecond-precision PTS values, with software fallback and ONE_SAMPLE_US monotonicity enforcement.
• Real-time audio regulation in audio_regulator.c maintains target buffering (~50ms default), inserts silence on under-run, and drops samples on overflow.
• FFmpeg resampling compensation via swr_set_compensation() continuously corrects clock drift between Android and host systems.
• Configurable buffers (--audio-buffer and --audio-output-buffer) allow users to tune the latency vs. stability trade-off.

Frequently Asked Questions

What is the default audio latency in scrcpy?

scrcpy targets approximately 50ms of audio latency by default, controlled by the --audio-buffer parameter. This represents the amount of audio buffered in the regulator before playback begins, balancing responsiveness with glitch resistance.

How does scrcpy prevent audio drift over time?

The audio regulator in audio_regulator.c uses FFmpeg's swr_set_compensation() to dynamically adjust the resampling ratio. By continuously comparing the expected PTS against the actual playback clock, it speeds up or slows down the audio stream in real-time to maintain synchronization with the video.

Can I reduce audio latency below 50ms in scrcpy?

Yes, you can reduce latency by passing a lower value to --audio-buffer, such as scrcpy --audio-buffer=30 for approximately 30ms. However, lower values increase vulnerability to audio glitches if the network or CPU experiences congestion, as the buffer has less time to recover from delayed packets.

Why does scrcpy use hardware timestamps instead of system time?

scrcpy prefers hardware timestamps from AudioRecord.getTimestamp() because they represent the exact moment samples were captured by the audio hardware, unaffected by Android system scheduling delays or Java VM latency. When hardware timestamps are unavailable, it falls back to software estimation while enforcing monotonicity to prevent backward PTS jumps.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}