Which Face Detection Method is Used Prior to PFLD? A Deep Dive into MTCNN Integration

The PFLD repository uses MTCNN (Multi-Task Cascaded Convolutional Networks) as the face detection method prior to PFLD, implementing a three-stage cascade in mtcnn/detect_face.py that feeds cropped face regions to the landmark regression network.

The guoqiangqi/pfld repository implements a Practical Facial Landmark Detector (PFLD) that requires a preceding face detection step to isolate facial regions before landmark prediction. Understanding which face detection method is used prior to PFLD is essential for reproducing the inference pipeline or modifying the detection backend.

What Face Detection Method is Used Before PFLD?

The repository employs MTCNN (Multi-Task Cascaded Convolutional Networks) as the dedicated face detection method prior to PFLD inference. MTCNN operates as a three-stage cascade architecture:

• P-Net (Proposal Network) – Generates candidate face bounding boxes quickly
• R-Net (Refine Network) – Refines the candidates by rejecting non-faces
• O-Net (Output Network) – Performs final face detection with higher accuracy

This cascade is implemented in mtcnn/detect_face.py, where the MTCNN class encapsulates the entire detection pipeline and exposes a predict() method for inference.

How MTCNN is Implemented in the PFLD Codebase

The MTCNN Class in detect_face.py

The core implementation resides in mtcnn/detect_face.py, which defines the MTCNN wrapper class. This class builds a TensorFlow computation graph containing the three-stage networks and loads pretrained weights from .npy files.

from mtcnn.detect_face import MTCNN
import cv2

# Instantiate the detector

detector = MTCNN()

# Read image

img = cv2.imread('input.jpg')

# Detect faces - returns list of [x1, y1, x2, y2, score]

boxes = detector.predict(img)

The predict() method returns bounding box coordinates in the format [x1, y1, x2, y2, score], where (x1, y1) represents the top-left corner, (x2, y2) the bottom-right corner, and score the detection confidence.

Loading Pretrained Weights

The MTCNN implementation loads three separate weight files located in the repository:

• det1.npy – Weights for P-Net
• det2.npy – Weights for R-Net
• det3.npy – Weights for O-Net

These NumPy archives contain the trained parameters for each cascade stage, enabling immediate inference without additional training.

Integrating MTCNN with PFLD: The Detection Pipeline

The live demonstration in camera.py illustrates the complete integration of MTCNN as the face detection method prior to PFLD landmark regression:

def main():
    # Initialize PFLD model session...

    cap = cv2.VideoCapture(0)
    mtcnn = MTCNN()  # Face detector instantiation

    
    while True:
        ret, image = cap.read()
        boxes = mtcnn.predict(image)  # Face detection step

        
        for box in boxes:
            x1, y1, x2, y2 = (box[:4] + 0.5).astype(np.int32)
            
            # Preprocessing: expand to square, pad, resize to 112×112

            cropped = preprocess_face(image, x1, y1, x2, y2)
            
            # PFLD landmark prediction

            landmarks = sess.run(
                landmarks_tensor,
                feed_dict={
                    images_placeholder: cropped,
                    phase_train_placeholder: False
                }
            )

This pipeline demonstrates that MTCNN serves as the mandatory preprocessing step, providing normalized face crops that the PFLD network expects as input.

Practical Code Examples

Running MTCNN Detection Standalone

To use the face detection method prior to PFLD in isolation:

from mtcnn.detect_face import MTCNN
import cv2
import numpy as np

# Initialize detector

detector = MTCNN()

# Load image

image = cv2.imread('portrait.jpg')

# Execute detection

bounding_boxes = detector.predict(image)

# Visualize results

for box in bounding_boxes:
    x1, y1, x2, y2, score = box
    if score > 0.7:  # Confidence threshold

        cv2.rectangle(
            image, 
            (int(x1), int(y1)), 
            (int(x2), int(y2)), 
            (0, 255, 0), 
            2
        )

cv2.imwrite('detected_faces.jpg', image)

Full Pipeline: Detection to Landmark Prediction

For complete integration where MTCNN feeds directly into PFLD:

import cv2
import numpy as np
from mtcnn.detect_face import MTCNN

def preprocess_face(image, x1, y1, x2, y2, target_size=112):
    """Crop, pad to square, and resize for PFLD input."""
    height, width = image.shape[:2]
    
    # Convert to square

    w, h = x2 - x1, y2 - y1
    size = max(w, h)
    cx, cy = (x1 + x2) // 2, (y1 + y2) // 2
    
    # New coordinates with padding check

    nx1 = max(0, cx - size // 2)
    ny1 = max(0, cy - size // 2)
    nx2 = min(width, cx + size // 2)
    ny2 = min(height, cy + size // 2)
    
    cropped = image[ny1:ny2, nx1:nx2]
    resized = cv2.resize(cropped, (target_size, target_size))
    return resized

# Usage

detector = MTCNN()
image = cv2.imread('input.jpg')
boxes = detector.predict(image)

for box in boxes:
    x1, y1, x2, y2, _ = box
    face_crop = preprocess_face(image, int(x1), int(y1), int(x2), int(y2))
    # Feed face_crop to PFLD model here

Summary

• MTCNN serves as the face detection method prior to PFLD in the guoqiangqi/pfld repository, implemented in mtcnn/detect_face.py.
• The detector uses a three-stage cascade (P-Net, R-Net, O-Net) with pretrained weights loaded from det1.npy, det2.npy, and det3.npy.
• The MTCNN class exposes a predict() method that returns bounding boxes [x1, y1, x2, y2, score], which camera.py uses to crop and preprocess faces before PFLD landmark regression.
• The pipeline requires converting detected boxes to square regions, padding to maintain aspect ratio, and resizing to 112×112 pixels for PFLD input compatibility.

Frequently Asked Questions

Why is MTCNN used as the face detection method prior to PFLD instead of other detectors?

MTCNN provides a robust balance between detection accuracy and computational efficiency through its cascaded architecture. The three-stage design (P-Net for fast proposal generation, R-Net for refinement, and O-Net for final detection with facial landmark alignment) ensures high recall rates for face detection while maintaining reasonable inference speed, making it suitable for real-time applications like the live demo in camera.py.

What input size does the face detection method output for PFLD?

While MTCNN outputs variable-sized bounding boxes depending on detected face dimensions, the preprocessing pipeline in camera.py standardizes these to 112×112 pixels before feeding them to the PFLD network. The code expands the detected bounding box to a square region, pads it if necessary to prevent border violations, and uses cv2.resize() to achieve the fixed input dimensions required by the PFLD model architecture.

Can I replace MTCNN with a different face detection method prior to PFLD?

Yes, you can substitute MTCNN with alternative detectors such as RetinaFace, YuNet, or MediaPipe, provided you maintain the output interface expected by the PFLD pipeline. Specifically, any replacement must return bounding box coordinates in the format [x1, y1, x2, y2] and include preprocessing logic to crop square regions resized to 112×112 pixels. You would modify camera.py to instantiate your chosen detector instead of MTCNN() while preserving the subsequent preprocessing and inference steps.

Where are the MTCNN model weights stored in the repository?

The pretrained weights for the three-stage cascade are stored as NumPy archive files in the repository root or model directory: det1.npy (P-Net weights), det2.npy (R-Net weights), and det3.npy (O-Net weights). The MTCNN class in mtcnn/detect_face.py loads these files during initialization to construct the TensorFlow computation graph for inference.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}