Deep-Live-Cam Face Analyzer Module: Structure and 106-Point 2D Landmarks Explained

The Deep-Live-Cam face analyzer module is a thread-safe singleton wrapper around InsightFace that extracts 106-point 2D facial landmarks from video frames, enabling precise geometric masking, alignment, and blending operations throughout the face-swapping pipeline.

The modules/face_analyser.py file serves as the central detection engine in the hacksider/Deep-Live-Cam repository. It transforms raw BGR frames into rich face objects containing high-resolution landmark data, which downstream processors leverage for everything from mouth-region isolation to face enhancement alignment. Understanding how this module structures its detection logic and how the 106-point landmark array flows through the system is essential for customizing or extending the application's computer vision capabilities.

Face Analyzer Module Architecture

Singleton Pattern and Model Initialization

The analyzer implements a thread-safe singleton pattern to ensure the computationally expensive InsightFace model loads only once across all processing threads. The get_face_analyser() function guards initialization with FACE_ANALYSER_LOCK, creating a single insightface.app.FaceAnalysis instance configured with the buffalo_l model and execution providers defined in modules/globals.py (CPU, CUDA, or CoreML).

# modules/face_analyser.py

def get_face_analyser():
    global FACE_ANALYSER
    with FACE_ANALYSER_LOCK:
        if FACE_ANALYSER is None:
            FACE_ANALYSER = insightface.app.FaceAnalysis(name='buffalo_l', ...)
            FACE_ANALYSER.prepare(ctx_id=0, det_size=(640, 640))
    return FACE_ANALYSER

Face Detection and Extraction Functions

The module provides two primary extraction interfaces that return InsightFace face objects containing the critical landmark_2d_106 attribute:

• get_one_face(frame) – Returns the left-most detected face by selecting min(..., key=lambda x: x.bbox[0]) from the bounding box coordinates.
• get_many_faces(frame) – Returns a list of all detected faces in the frame via the underlying analyzer.

Each face object exposes bbox (bounding box coordinates), normed_embedding (512-dimensional feature vector for clustering), and landmark_2d_106 (the 106-point 2D array).

Source-Target Map Building

For batch processing of images or videos, the analyzer builds a source-target mapping structure stored in modules.globals.source_target_map. The functions get_unique_faces_from_target_image() and get_unique_faces_from_target_video() populate this map by:

1. Walking through every frame of the target media.
2. Extracting faces and their embeddings.
3. Clustering embeddings via modules/cluster_analysis.py to create centroids for consistent identity tracking across video frames.

Additional utilities like add_blank_map(), simplify_maps(), and has_valid_map() manage the mapping state, while default_source_face() retrieves the primary source face for swapping operations.

How 106-Point 2D Landmarks Are Utilized

The landmark_2d_106 array provides a high-resolution geometric representation compared to standard 5-point landmarks. Downstream processors slice this array to isolate specific facial regions for targeted operations.

Landmark-Based Mask Generation

In modules/processors/frame/face_masking.py, the full 106-point set drives the creation of feathered facial masks. The processor slices the array into anatomical regions: indices 0-32 define the face outline, 33-51 and 97-105 define eyebrows, and specific ranges handle eyes and lips.

# modules/processors/frame/face_masking.py

landmarks = face.landmark_2d_106
if landmarks is not None:
    landmarks = landmarks.astype(np.int32)
    face_outline = landmarks[0:33]          # 0-32 → face contour

    eyebrows = landmarks[33:43] + landmarks[97:105]
    # ... create mask with cv2.fillPoly(face_outline) ...

Mouth Region Isolation and Blending

The modules/processors/frame/face_swapper.py processor validates the presence of landmark_2d_106 before generating mouth-specific masks. It extracts indices 52-63 to build a lower-lip polygon, expands the landmarks outward for blending margins, and uses the coordinates for Poisson blending operations.

# modules/processors/frame/face_swapper.py

if face is None or not hasattr(face, 'landmark_2d_106'):
    return mask, mouth_cutout, mouth_box, lower_lip_polygon

landmarks = face.landmark_2d_106
lower_lip_order = list(range(52, 64))                     # outer-mouth points

lower_lip_landmarks = landmarks[lower_lip_order].astype(np.float32)
center = np.mean(lower_lip_landmarks, axis=0)
expanded_landmarks = (lower_lip_landmarks - center) * (1 + mask_down_size) + center
expanded_landmarks = expanded_landmarks.astype(np.int32)

Geometric Alignment for Enhancement

The face enhancer pipeline in modules/processors/frame/_onnx_enhancer.py utilizes the 106-point landmarks for affine transformation calculations. When aligning faces for super-resolution models, it either uses the standard 5-point landmarks (face.kps) or falls back to the 106-point set, computing the transformation matrix via cv2.estimateAffinePartial2D to standardize face orientation before enhancement.

Practical Implementation Examples

Extracting Faces and Accessing 106-Point Landmarks

To retrieve a single face and its landmark array from a static image:

from modules.face_analyser import get_one_face
import cv2

frame = cv2.imread("target.jpg")          # any BGR frame

face = get_one_face(frame)

if face and hasattr(face, "landmark_2d_106"):
    landmarks = face.landmark_2d_106      # shape (106, 2)

    # Draw the outer-mouth polygon (indices 52-63)

    mouth_idxs = list(range(52, 64))
    mouth_pts = landmarks[mouth_idxs].astype(int)
    cv2.polylines(frame, [mouth_pts], isClosed=True, color=(0,255,0), thickness=2)
    cv2.imwrite("annotated.jpg", frame)

Creating Full-Face Masks from Landmark Data

Generate a binary mask using the analyzer's face object:

from modules.face_analyser import get_one_face
from modules.processors.frame.face_swapper import create_face_mask
import cv2

frame = cv2.imread("target.jpg")
face = get_one_face(frame)

mask = create_face_mask(face, frame)   # uint8 mask (0-255)

masked_frame = cv2.bitwise_and(frame, frame, mask=mask)
cv2.imwrite("masked_output.png", masked_frame)

Processing Video Targets with Face Clustering

For video processing, populate the global source-target map to enable frame-by-frame identity tracking:

from modules.face_analyser import get_unique_faces_from_target_video
from modules.globals import target_path, source_target_map

# Analyzes every frame, clusters embeddings, and fills source_target_map

get_unique_faces_from_target_video()

# Each entry in source_target_map now contains:

# - Face objects with landmark_2d_106

# - normed_embedding vectors

# - Centroid data for consistent identity matching

Summary

• The face analyzer module (modules/face_analyser.py) implements a thread-safe singleton that wraps the InsightFace buffalo_l model, providing get_one_face() and get_many_faces() extraction functions.
• Every detected face object contains a landmark_2d_106 attribute—a NumPy array of shape (106, 2) representing high-resolution facial geometry in image coordinates.
• Downstream processors slice these landmarks into specific regions: indices 0-32 for the face contour, 52-63 for the lower lip/mouth, and 33-51/97-105 for eyebrows.
• The source-target map system uses face embeddings (normed_embedding) and 106-point landmarks to maintain consistent identity tracking and geometric alignment across video frames.
• Mask generation, Poisson blending, and affine alignment for enhancement all depend on this 106-point landmark data structure.

Frequently Asked Questions

What is the face analyzer module in Deep-Live-Cam?

The face analyzer module is the central detection component located in modules/face_analyser.py. It wraps the InsightFace library to provide thread-safe face detection, extracting rich face objects that include bounding boxes, 512-dimensional embedding vectors, and 106-point 2D landmarks used throughout the swapping and enhancement pipeline.

How are the 106-point landmarks different from standard 5-point landmarks?

While the 5-point landmarks (face.kps) provide basic eye and nose positions for rough alignment, the 106-point 2D landmarks (landmark_2d_106) offer a dense geometric mesh covering the full face contour, eyebrows, eyes, nose, and mouth. This higher resolution enables precise polygon-based masking for mouth-region blending and detailed facial segmentation in face_masking.py and face_swapper.py.

Which files consume the landmark_2d_106 data?

Three primary processors consume the 106-point landmark array: modules/processors/frame/face_swapper.py (mouth masks and blending), modules/processors/frame/face_masking.py (full-face feathered masks), and modules/processors/frame/_onnx_enhancer.py (geometric alignment for super-resolution). Each slices the array differently to isolate specific facial regions.

How does the module handle multiple faces in video processing?

For video targets, get_unique_faces_from_target_video() extracts faces from every frame, stores them in modules.globals.source_target_map, and clusters the normed_embedding vectors using modules/cluster_analysis.py. This creates centroids that track consistent identities across frames, allowing the 106-point landmarks to be matched to the correct source face throughout the video sequence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}