Faceswap Masker Plugins: Available Types and How They Work

Faceswap provides seven masker plugins—Components, Extended, Custom, VGG Clear, VGG Obstructed, U-Net DFL, and BiSeNet Face Parsing—that generate facial masks using either geometric landmark analysis or pre-trained neural networks to isolate facial regions during the extraction pipeline.

The deepfakes/faceswap repository contains a modular masking subsystem in plugins/extract/mask/ that controls which facial regions are isolated during face extraction. These masker plugins inherit from a common Masker base class and can be selected via the CLI or Python API to optimize masking for different face angles, occlusions, or editing workflows. Understanding how these masker plugins function allows you to choose between lightweight geometric masks and deep-learning segmentation models.

Available Masker Plugins in Faceswap

The repository offers seven distinct masker plugins, categorized into geometric landmark-based approaches and neural-network segmentation models.

Geometric Landmark-Based Maskers

Components – This plugin creates masks by building convex hulls around specific facial parts defined by 68-point landmarks. In plugins/extract/mask/components.py, the parse_parts() method groups landmarks for the jaw, cheeks, eyes, and nose, then uses cv2.fillConvexPoly to fill these regions with 1.0 values. No neural network weights are loaded, leaving init_model() unimplemented.

Extended – Located in plugins/extract/mask/extended.py, this plugin inherits from Components but first calls _adjust_mask_top() to raise the landmark boundary upward to include eyebrow regions. It then applies the same convex hull filling logic, extending coverage to the upper face.

Custom – Implemented in plugins/extract/mask/custom.py, this plugin generates a full-face or full-head binary mask filled entirely with 1.0 or 0.0 values based on user configuration in custom_defaults.py. This requires no geometric calculations or model inference, making it ideal for manual mask editing workflows.

Neural-Network Based Maskers

VGG Clear – Defined in plugins/extract/mask/vgg_clear.py, this plugin loads the pre-trained Nirkin_300_softmax_v1.h5 model and processes 300×300 pixel inputs through a VGG-based fully convolutional network (FCN). It outputs a 2-class softmax prediction distinguishing face from background.

VGG Obstructed – As implemented in plugins/extract/mask/vgg_obstructed.py, this variant uses the Nirkin_500_softmax_v1.h5 model with 500×500 inputs. The architecture matches VGG Clear but is trained specifically on images containing occlusions and partial face coverage.

U-Net DFL – This plugin in plugins/extract/mask/unet_dfl.py employs a TernausNet-style UNet architecture loaded from DFL_256_sigmoid_v1.h5. It processes 256×256 inputs and produces binary face masks via sigmoid activation.

BiSeNet Face Parsing – Located in plugins/extract/mask/bisenet_fp.py, this plugin loads bisnet_face_parsing_v*.h5 and processes 512×512 inputs through a BiSeNet architecture. Unlike binary maskers, it supports multi-class semantic segmentation including skin, hair, glasses, and ears based on configuration in bisenet_fp_defaults.py.

How Masker Plugins Work

All masker plugins inherit from the Masker base class in plugins/extract/mask/_base.py and implement a standardized four-stage processing pipeline.

The Masker Base Class Architecture

The abstract Masker class initializes with git_model_id and model_filename parameters to handle automatic model downloading from the Faceswap model repository. According to the source code in _base.py, subclasses must override four critical methods:

• init_model() – Loads neural network weights or initializes geometric calculators.
• process_input(batch) – Transforms raw face images into normalized tensors for inference.
• predict(feed) – Executes model inference or geometric logic to generate raw mask data.
• process_output(batch) – Converts predictions into final binary float32 masks stored in the batch object.

Geometric Masking Implementation

Geometric maskers bypass neural networks entirely. In plugins/extract/mask/components.py, the plugin iterates through landmark subsets (jaw indices 0-16, eyebrows, etc.), calculates convex hulls using OpenCV, and fills polygons to create the mask. The Extended plugin in extended.py overrides initialization to adjust eyebrow landmarks upward before applying the same hull-filling logic via cv2.fillConvexPoly.

Neural-Network Masking Pipeline

Neural-network maskers in vgg_clear.py, vgg_obstructed.py, unet_dfl.py, and bisenet_fp.py follow a consistent execution pattern:

1. Model Initialization – init_model() instantiates architecture-specific wrappers (VGGClear, UnetDFL, BiSeNet) that build Keras models and load weights.
2. Input Processing – process_input() normalizes images via mean/std subtraction and optional color space conversion.
3. Inference – predict() calls the wrapper's __call__ method, returning per-pixel class probabilities.
4. Output Processing – process_output() filters for foreground classes (or specific segment indices in BiSeNet) and compresses the result to a single-channel float32 array.

Using Masker Plugins in Practice

You can invoke masker plugins via the command line for standard extraction workflows or programmatically through the Python API for custom pipelines.

Command Line Selection

Use the -M or --masker flag to specify one or more maskers during extraction:

python faceswap.py extract -i input_folder -o output_folder -M components extended

This command runs both the Components and Extended geometric maskers simultaneously, generating separate mask channels for each face.

Programmatic Usage

Access maskers through PluginLoader for custom Python scripts:

from plugins.plugin_loader import PluginLoader
from lib.align import AlignedFace

# Load the VGG Clear masker class

MaskerCls = PluginLoader.get_masker('vgg_clear')
masker = MaskerCls(configfile='config.ini')  # Auto-downloads weights if needed

# Prepare a batch with aligned faces

batch.feed_faces = [AlignedFace(face_image_array)]  # numpy array (H, W, 3)

# Execute the pipeline

masker.process_input(batch)      # Prepares batch.feed tensor

raw_predictions = masker.predict(batch.feed)
masker.process_output(batch)     # Stores result in batch.prediction

final_mask = batch.prediction    # Binary float32 mask array

To use multiple maskers together, instantiate each via PluginLoader.get_masker() and process batches sequentially, or use Faceswap's internal pipeline helpers which return dictionaries mapping masker names to mask arrays.

Summary

• Seven masker plugins are available in plugins/extract/mask/: Components, Extended, Custom, VGG Clear, VGG Obstructed, U-Net DFL, and BiSeNet Face Parsing.
• Geometric maskers (Components, Extended, Custom) leverage 68-point landmarks and OpenCV convex hulls without requiring neural network inference.
• Neural maskers load pre-trained Keras models with specific input resolutions: VGG models (300×300 and 500×500), U-Net DFL (256×256), and BiSeNet (512×512).
• All plugins inherit from the Masker base class in plugins/extract/mask/_base.py and implement standardized init_model(), process_input(), predict(), and process_output() methods.
• Selection occurs via the -M CLI flag or PluginLoader.get_masker(name) for Python integration.
• BiSeNet uniquely supports multi-class segmentation (hair, glasses, ears, skin) while other maskers produce binary face/background masks.

Frequently Asked Questions

What is the difference between VGG Clear and VGG Obstructed masker plugins?

VGG Clear uses a 300×300 input VGG-based FCN trained on clear, unobstructed faces using Nirkin_300_softmax_v1.h5, while VGG Obstructed uses a 500×500 input model (Nirkin_500_softmax_v1.h5) specifically trained to handle challenging images with occlusions. Both plugins are implemented in plugins/extract/mask/vgg_clear.py and vgg_obstructed.py respectively and output 2-class softmax predictions, but the Obstructed variant provides better coverage when faces are partially covered by objects or hands.

How do Components and Extended masker plugins differ?

Both plugins use geometric convex hulls around 68-point facial landmarks, but Extended specifically adjusts the mask boundary upward to include eyebrow regions. According to plugins/extract/mask/extended.py, the plugin calls _adjust_mask_top() to lift eyebrow points before applying the same cv2.fillConvexPoly filling logic used in Components. Use Extended when you need the mask to cover the upper face and eyebrows, or Components for a tighter jawline-focused outline.

Can I use multiple masker plugins simultaneously during extraction?

Yes. The CLI accepts multiple masker names via the -M or --masker flag (e.g., -M components vgg_clear bisenet_fp). Each plugin generates its own mask channel, allowing you to compare geometric versus neural-network approaches or combine masks for different facial features, such as using Components for the jawline and BiSeNet for hair segmentation.

Do I need to download neural network weights manually for masker plugins?

No. The Masker base class in plugins/extract/mask/_base.py handles automatic weight downloading via the git_model_id and model_filename parameters. When you instantiate a neural-network masker like VGG Clear or BiSeNet, the init_model() method automatically fetches the required .h5 files (such as Nirkin_300_softmax_v1.h5 or bisnet_face_parsing_v*.h5) from the Faceswap model repository if they are not present locally.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}