# Deep-Live-Cam Mouth Mask Feature: How Lower Lip Landmarks Drive Face Swap Isolation

> Discover how Deep Live Cam uses lower lip landmarks to isolate mouths for face swaps. Learn about polygon expansion, GPU blurring, and feathered masking for seamless integration.

- Repository: [Kenneth Estanislao/Deep-Live-Cam](https://github.com/hacksider/Deep-Live-Cam)
- Tags: deep-dive
- Published: 2026-03-01

---

**The mouth mask feature isolates the lower lip using landmarks 52-63 from the 106-point facial landmark set, expands the polygon via configurable scaling factors, and blends swapped mouths back into frames using GPU-accelerated Gaussian blur and feathered masking.**

The **Deep-Live-Cam** repository implements real-time face swapping with surgical precision by targeting specific facial regions. The **mouth mask feature** zeroes in on the lower lip area using precise **lower lip landmarks** to ensure seamless mouth replacement without affecting surrounding facial features. This implementation relies on the 106-point landmark model and configurable expansion parameters defined in the codebase.

## Landmark-Based Isolation in face_masking.py

The core algorithm lives in [`modules/processors/frame/face_masking.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/processors/frame/face_masking.py), where the `create_lower_mouth_mask` function processes detected faces to generate binary masks. The system extracts the **106-point facial landmark** array from the face analyser and immediately filters for the outer mouth contour.

### Selecting Lower Lip Landmarks 52-63

The algorithm targets indices **52 through 63** (inclusive) from the landmark array, which correspond exclusively to the outer lip contour. This selection excludes chin and jaw points, ensuring the mask covers only the mouth region.

```python
landmarks = face.landmark_2d_106  # face_masking.py#L83-L86

lower_lip_order = list(range(52, 64))  # face_masking.py#L85-L87

lower_lip_landmarks = landmarks[lower_lip_order].astype(np.float32)  # face_masking.py#L91-L92

```

### Geometric Center and Expansion Logic

After isolating the twelve lower lip points, the code calculates the arithmetic mean to find the polygon center. It then scales each point outward using the formula `1 + mask_down_size * mouth_mask_size`, creating a safety margin around the lip.

```python
center = np.mean(lower_lip_landmarks, axis=0)  # face_masking.py#L94-L95

expansion_factor = (
    1 + modules.globals.mask_down_size * modules.globals.mouth_mask_size
)  # face_masking.py#L98-L100

expanded_landmarks = (lower_lip_landmarks - center) * expansion_factor + center
expanded_landmarks = expanded_landmarks.astype(np.int32)  # face_masking.py#L101-L106

```

## Building the Binary Mask and ROI

With the expanded polygon defined, the system constructs a binary mask limited to the mouth region. This process involves calculating padded bounding boxes and drawing filled polygons relative to the region of interest.

### Padded Bounding Box Calculation

The code derives the tight bounding box from the expanded landmarks and adds **10% padding** relative to the mask width. This prevents edge clipping during the blending phase.

```python
min_x, min_y = np.min(expanded_landmarks, axis=0)  # face_masking.py#L108-L109

max_x, max_y = np.max(expanded_landmarks, axis=0)  # face_masking.py#L110-L111

padding = int((max_x - min_x) * 0.1)  # face_masking.py#L112-L114

min_x = max(0, min_x - padding)  # face_masking.py#L115-L116

min_y = max(0, min_y - padding)
max_x = min(frame.shape[1], max_x + padding)  # face_masking.py#L117-L118

max_y = min(frame.shape[0], max_y + padding)

```

### Polygon Drawing and ROI Mask Generation

The algorithm creates a blank ROI mask sized to the padded bounding box, translates the expanded landmarks to ROI-relative coordinates, and fills the polygon with white (255).

```python
mask_roi = np.zeros((max_y - min_y, max_x - min_x), dtype=np.uint8)  # face_masking.py#L126-L127

polygon_relative_to_roi = expanded_landmarks - [min_x, min_y]  # face_masking.py#L129-L130

cv2.fillPoly(mask_roi, [polygon_relative_to_roi], 255)  # face_masking.py#L131-L132

```

## GPU-Accelerated Edge Feathering

To eliminate hard edges during blending, the mouth mask feature applies **GPU-accelerated Gaussian blur** to the binary mask. The implementation uses a 15-pixel kernel with sigma 5 to create a soft alpha gradient.

```python
mask_roi = gpu_gaussian_blur(mask_roi, (15, 15), 5)  # face_masking.py#L132-L134

mask[min_y:max_y, min_x:max_x] = mask_roi  # face_masking.py#L135-L136

```

This feathering ensures the swapped mouth integrates naturally with the original skin texture and lighting conditions.

## Extracting and Reinserting the Mouth Cutout

The completed mask enables surgical extraction and replacement of mouth regions without disturbing the broader face structure.

### Original Mouth Patch Extraction

Before swapping occurs, the system preserves the original mouth pixels using the calculated bounding box.

```python
mouth_cutout = frame[min_y:max_y, min_x:max_x].copy()  # face_masking.py#L138-L139

```

### Blending with apply_mask_area

During the face swap pipeline, the `apply_mask_area` function (defined at `face_masking.py#L21-L50`) resizes the swapped mouth to fit the ROI, performs color transfer, and blends it back using the feathered polygon mask combined with the global face mask.

The function returns four critical artifacts:
- **mask**: Full-frame binary mask
- **mouth_cutout**: Original pixel data for the mouth region  
- **box**: Tuple of (xmin, ymin, xmax, ymax) coordinates
- **expanded_landmarks**: Polygon vertices in original frame coordinates

## Configuration Parameters in globals.py

The expansion behavior is controlled by two global parameters defined in [`modules/globals.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/globals.py):
- **mask_down_size**: Base expansion factor (default 0.1)
- **mouth_mask_size**: User-tunable multiplier (default 1.0)

These values multiply to determine the final polygon expansion, allowing fine-tuning for different face shapes and mouth sizes.

## Code Example: Implementing the Mouth Mask Pipeline

The following snippet demonstrates direct usage of the lower lip mask API outside the automatic face swap processor:

```python
import cv2
from modules.processors.frame.face_masking import create_lower_mouth_mask
from modules.processors.frame.face_masking import apply_mask_area
import modules.globals

# Configure expansion (optional)

modules.globals.mask_down_size = 0.1
modules.globals.mouth_mask_size = 1.2

# Generate mask from detected face

mask, cutout, box, polygon = create_lower_mouth_mask(detected_face, frame)

# Assume new_mouth contains the swapped texture

result = apply_mask_area(
    frame=frame,
    cutout=new_mouth,
    box=box,
    face_mask=mask,
    polygon=polygon
)

cv2.imwrite("swapped_output.png", result)

```

## Summary

- The mouth mask feature targets **landmarks 52-63** from the 106-point facial landmark set to isolate the lower lip exclusively.
- Polygon expansion uses the formula `1 + mask_down_size * mouth_mask_size` to create configurable safety margins around the mouth.
- **10% padding** is added to the bounding box to prevent edge clipping during extraction.
- GPU-accelerated Gaussian blur with a 15x15 kernel and sigma 5 feathers the mask edges for seamless blending.
- The primary implementation resides in [`modules/processors/frame/face_masking.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/processors/frame/face_masking.py), with duplicate logic in [`face_swapper.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/face_swapper.py).
- The `apply_mask_area` function handles color transfer and final compositing of the swapped mouth region.

## Frequently Asked Questions

### Which landmark indices define the lower lip mask in Deep-Live-Cam?

The lower lip mask exclusively uses landmarks **52 through 63** from the 106-point facial landmark array. These indices correspond to the outer lip contour, deliberately excluding chin and jaw points to restrict the mask strictly to the mouth region.

### How does the mouth_mask_size parameter affect the mask?

The **mouth_mask_size** parameter acts as a user-tunable multiplier that scales the base expansion factor. Combined with **mask_down_size** (default 0.1) via the formula `1 + mask_down_size * mouth_mask_size`, it controls how far the polygon expands outward from the geometric center of the lower lip landmarks.

### Why is GPU Gaussian blur used instead of CPU blur?

The implementation calls **gpu_gaussian_blur** with a 15x15 kernel and sigma 5 to accelerate the feathering operation. This GPU acceleration ensures real-time performance during face swapping while softening the binary mask edges to prevent visible seams between the original frame and the swapped mouth region.

### Where is the mouth mask logic duplicated besides face_masking.py?

An identical implementation of the lower lip masking logic exists in [`modules/processors/frame/face_swapper.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/processors/frame/face_swapper.py) (lines 32-42), where it performs the same landmark selection and expansion during the active face swap processing pipeline, though [`face_masking.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/face_masking.py) contains the canonical version with the full feathering and extraction pipeline.