How Deep-Live-Cam Handles Audio Restoration After Video Processing

Deep-Live-Cam restores audio by using ffmpeg to copy the processed video stream and map the original audio track from the source file, ensuring the final output retains the original soundtrack without re-encoding the video.

The open-source face-swapping application Deep-Live-Cam (hacksider/Deep-Live-Cam) processes video frames independently from audio streams to preserve quality and avoid synchronization issues. After AI-driven frame processing completes in modules/core.py, the application seamlessly reattaches the original audio using a targeted ffmpeg command. This modular approach ensures that users receive a complete video file with processed visuals intact alongside the untouched audio track.

The Audio Restoration Pipeline

Deep-Live-Cam separates video frame processing from audio handling through a distinct two-phase workflow.

First, the create_video() function in modules/core.py processes individual frames (face-swapping, enhancement, etc.) and re-encodes them into a temporary video file containing only the video stream. This temporary file excludes audio entirely to prevent codec conflicts during the intensive frame manipulation phase.

Second, if the user has enabled the "Keep audio" option, the pipeline invokes restore_audio() from modules/utilities.py. This function executes a precise ffmpeg command that merges the processed video with the original audio stream without re-encoding either track.

Configuration Options for Audio Handling

The Deep-Live-Cam audio restoration feature is optional and controlled through both command-line arguments and the graphical interface.

Command-Line Interface

In modules/core.py (lines 44-46), the application defines the --keep-audio argument that defaults to True. Users can explicitly disable audio restoration:

python run.py --keep-audio false --source input.mp4 --target face.png --output result.mp4

When omitted, the default behavior preserves the audio track in the final output.

Graphical User Interface

For UI users, modules/ui.py (lines 124-128) provides a "Keep audio" toggle switch that modifies the global modules.globals.keep_audio variable. This boolean flag (defined in modules/globals.py with default value True) determines whether the pipeline executes the restoration step after frame processing completes.

Technical Implementation of restore_audio()

The actual audio merging logic resides in modules/utilities.py (lines 92-110) within the restore_audio() function. This implementation uses ffmpeg's stream mapping capabilities to combine files efficiently:

def restore_audio(target_path: str, output_path: str) -> None:
    temp_output_path = get_temp_output_path(target_path)
    done = run_ffmpeg([
        "-i", temp_output_path,       # Processed video without audio (input 0)

        "-i", target_path,            # Original video with audio (input 1)

        "-c:v", "copy",               # Copy video codec without re-encoding

        "-map", "0:v:0",              # Select video stream from first input

        "-map", "1:a:0",              # Select audio stream from second input

        "-y", output_path,            # Force overwrite output

    ])
    if not done:
        move_temp(target_path, output_path)  # Fallback: video without audio

Key technical details:

• -c:v copy: Preserves the exact video encoding from the processed temp file, maintaining frame quality.
• -map 0:v:0: Selects the video stream from the temporary processed file.
• -map 1:a:0: Selects the audio stream from the original source video (target_path).
• temp_output_path: Generated by get_temp_output_path(target_path), this points to the video-only file created during frame processing.

Error Handling and Fallbacks

If the ffmpeg command fails—typically due to timestamp mismatches or codec incompatibilities between the processed video and original audio—Deep-Live-Cam implements a graceful degradation strategy. The restore_audio() function checks the return status of run_ffmpeg(). When the operation fails (returns False), the code falls back to move_temp(), which simply moves the temporary video file to the final output destination.

This ensures users always receive their processed video, even if audio restoration encounters technical issues. The resulting file contains the processed frames but lacks the audio track.

Summary

• Deep-Live-Cam separates video frame processing from audio handling by creating a temporary video-only file during processing.
• The keep_audio global flag (default True in modules/globals.py) controls whether audio restoration occurs after processing.
• The restore_audio() function in modules/utilities.py executes an ffmpeg command using -map to combine the processed video stream with the original audio stream.
• Both the --keep-audio CLI flag and the UI toggle in modules/ui.py allow users to disable this feature.
• If audio restoration fails, the system falls back to outputting the video without audio rather than failing entirely.

Frequently Asked Questions

Does Deep-Live-Cam re-encode the audio during restoration?

No. The ffmpeg command in restore_audio() uses stream mapping (-map 1:a:0) to copy the audio track directly from the original source without re-encoding. The video stream is also copied (-c:v copy) rather than re-encoded, preserving the exact quality of the processed frames.

What happens if ffmpeg fails to restore the audio?

If the ffmpeg command encounters errors such as mismatched timestamps between the processed video and original audio, Deep-Live-Cam falls back to the move_temp() function. This moves the temporary video file (which lacks audio) to the final output path, ensuring the user still receives the processed video even if audio restoration fails.

Can I restore audio from a different source file?

No. According to the implementation in modules/utilities.py, the restore_audio() function specifically maps the audio stream from the original target_path provided during processing. The function signature and ffmpeg command structure do not support substituting a different audio source file.

Why is there no audio in my output video despite using the default settings?

First, verify that your original input video actually contains an audio track. Then confirm that the keep_audio flag has not been disabled via the UI checkbox in modules/ui.py or the --keep-audio false CLI argument. If the ffmpeg merge operation fails due to codec incompatibilities or timestamp alignment issues, Deep-Live-Cam will output the processed video without audio as a fallback mechanism.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}