# How to Integrate PFLD with Other Face Analysis Tasks: A Complete Pipeline Guide

> Integrate PFLD with emotion recognition face alignment and attribute classification. This guide provides a complete pipeline for seamless integration of PFLD with downstream face analysis tasks.

- Repository: [Guoqiang QI/pfld](https://github.com/guoqiangqi/pfld)
- Tags: how-to-guide
- Published: 2026-03-03

---

**PFLD provides a compact TensorFlow model that outputs 98 facial landmarks and 3D head pose angles, enabling plug-and-play integration with downstream face analysis tasks such as emotion recognition, face alignment, and attribute classification.**

The guoqiangqi/pfld repository implements a lightweight Progressive Face Localization and Detection (PFLD) architecture that separates landmark detection from pose estimation. This dual-output design produces geometrically consistent facial features that you can feed directly into specialized analyzers without retraining the backbone network.

## Understanding PFLD's Dual-Output Architecture

The model is designed specifically for integration, exposing two distinct tensors through a shared MobileNet-V2 backbone defined in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py).

### Facial Landmark Heat-Maps (196-Dimensional Vector)

The `pfld_inference()` function in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) (lines 11-90) extracts intermediate feature maps and feeds them into a multi-scale fully-connected head. This produces `landmarks_pre`, a 196-dimensional vector that reshapes to 98 (x, y) coordinate pairs representing facial keypoints.

### Head Pose Euler Angles (3-Dimensional Vector)

An auxiliary branch sharing early feature maps regresses head orientation via `euler_angles_pre`. Implemented in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) (lines 118-138), this sub-network outputs yaw, pitch, and roll angles without requiring a separate pose estimation model.

## Step-by-Step Integration Pipeline

Integrating PFLD into a broader face analysis system follows a five-stage workflow using only TensorFlow, NumPy, and OpenCV operations.

### 1. Face Detection with MTCNN

The repository includes a standalone MTCNN implementation in [`mtcnn/detect_face.py`](https://github.com/guoqiangqi/pfld/blob/main/mtcnn/detect_face.py). Use this to generate bounding boxes before cropping:

```python
from mtcnn import detect_face

detectors = detect_face.create_mtcnn(sess, None)
bounding_boxes, _ = detect_face.detect_face(
    frame, minsize=20,
    thresholds=[0.6, 0.7, 0.7],
    factor=0.709,
    pnet=detectors[0],
    rnet=detectors[1],
    onet=detectors[2]
)

```

### 2. Preprocessing for PFLD Input

Extract each face region and resize to the model's expected input dimensions. PFLD requires 112×112 RGB images normalized to [0, 1]:

```python
face = frame[y1:y2, x1:x2]
face_rgb = cv2.cvtColor(cv2.resize(face, (112, 112)), cv2.COLOR_BGR2RGB)
face_rgb = face_rgb.astype(np.float32) / 256.0
face_rgb = np.expand_dims(face_rgb, 0)  # Add batch dimension

```

### 3. Model Inference

Load the pretrained checkpoint and retrieve output tensors by their fixed graph names. The landmark output is `fc:0` and the pose vector is `pfld_fc2:0`:

```python

# Load graph

saver = tf.train.import_meta_graph('./models2/model0/model.meta')
saver.restore(sess, './models2/model0/model.ckpt-0')

# Get tensors

img_ph = graph.get_tensor_by_name('image_batch:0')
phase_ph = graph.get_tensor_by_name('phase_train:0')
landmarks_tensor = graph.get_tensor_by_name('fc:0')      # 98 points

pose_tensor = graph.get_tensor_by_name('pfld_fc2:0')   # Euler angles

# Run inference

landmarks_vec, pose_vec = sess.run(
    [landmarks_tensor, pose_tensor],
    feed_dict={img_ph: face_rgb, phase_ph: False}
)

```

### 4. Post-Processing Landmarks

Reshape the 196-dimensional vector into coordinate pairs and denormalize to the original image scale:

```python
landmarks = landmarks_vec.reshape(-1, 2) * np.array([face.shape[1], face.shape[0]])

```

### 5. Feeding Downstream Modules

Pass the processed landmarks and pose data to specialized analyzers. The separation of concerns allows you to swap detectors or classifiers while keeping PFLD as a fixed geometric extractor.

## Connecting PFLD to Specific Downstream Tasks

The 98-point landmark set and pose vector support multiple analysis pipelines:

- **Face Alignment**: Convert landmarks into a similarity transform using key points (e.g., eye centers and nose tip) to warp faces to a canonical view before recognition.
- **Head-Pose Estimation**: Use `euler_angles_pre` directly for gaze tracking or driver monitoring, or fuse it with geometric solvers for higher precision.
- **Facial Expression Recognition**: Feed aligned face crops (generated via landmark-based affine transforms) into emotion classification CNNs.
- **Face Recognition**: Normalize face orientation using the predicted geometry before extracting embeddings from deep recognition networks.
- **Attribute Analysis**: Define region-of-interest masks (mouth, eyes, cheeks) using specific landmark indices to focus attribute classifiers on relevant facial zones.

## Complete Integration Code Example

The following snippet demonstrates a full detect-align-analyze pipeline combining MTCNN detection, PFLD inference, and downstream emotion classification:

```python
import tensorflow as tf
import numpy as np
import cv2
from mtcnn import detect_face

# Initialize session and load PFLD

graph = tf.Graph()
with graph.as_default():
    sess = tf.Session()
    saver = tf.train.import_meta_graph('./models2/model0/model.meta')
    saver.restore(sess, './models2/model0/model.ckpt-0')
    
    img_ph = graph.get_tensor_by_name('image_batch:0')
    phase_ph = graph.get_tensor_by_name('phase_train:0')
    landmarks_tensor = graph.get_tensor_by_name('fc:0')
    pose_tensor = graph.get_tensor_by_name('pfld_fc2:0')

detectors = detect_face.create_mtcnn(sess, None)

def align_face(img, landmarks):
    """Affine transform using left eye (36), right eye (45), nose tip (30)"""
    pts_src = np.float32([landmarks[36], landmarks[45], landmarks[30]])
    pts_dst = np.float32([[30, 30], [82, 30], [56, 70]])
    M = cv2.getAffineTransform(pts_src, pts_dst)
    return cv2.warpAffine(img, M, (112, 112))

def process_frame(frame):
    # Detect faces

    boxes, _ = detect_face.detect_face(frame, 20, [0.6, 0.7, 0.7], 0.709, 
                                      detectors[0], detectors[1], detectors[2])
    
    for bbox in boxes:
        x1, y1, x2, y2 = map(int, bbox[:4])
        face_crop = frame[y1:y2, x1:x2]
        
        # Preprocess

        face_rgb = cv2.cvtColor(cv2.resize(face_crop, (112, 112)), cv2.COLOR_BGR2RGB)
        face_rgb = face_rgb.astype(np.float32) / 256.0
        face_rgb = np.expand_dims(face_rgb, 0)
        
        # Inference

        lm_vec, pose = sess.run([landmarks_tensor, pose_tensor],
                               feed_dict={img_ph: face_rgb, phase_ph: False})
        
        # Post-process

        landmarks = lm_vec.reshape(-1, 2) * np.array([face_crop.shape[1], face_crop.shape[0]])
        
        # Downstream: Align and classify emotion

        aligned = align_face(face_crop, landmarks)
        emotion = emotion_classifier.predict(aligned)  # Your model here

        
        print(f"Pose (yaw, pitch, roll): {pose}")
        print(f"Detected emotion: {emotion}")

# Run on video

cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()
    if not ret:
        break
    process_frame(frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()

```

## Key Source Files for Integration

| File | Role |
|------|------|
| [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) | Defines the MobileNet-V2 backbone, multi-scale landmark head (`pfld_inference`), and auxiliary pose branch. |
| [`mtcnn/detect_face.py`](https://github.com/guoqiangqi/pfld/blob/main/mtcnn/detect_face.py) | Stand-alone MTCNN face detector for obtaining bounding boxes before PFLD processing. |
| [`utils.py`](https://github.com/guoqiangqi/pfld/blob/main/utils.py) | Implements `LandmarkImage` and `LandmarkImage_98` for converting coordinates to Gaussian heat-maps. |
| [`test_model.py`](https://github.com/guoqiangqi/pfld/blob/main/test_model.py) | Reference inference script demonstrating checkpoint loading and visualization. |
| [`train_model.py`](https://github.com/guoqiangqi/pfld/blob/main/train_model.py) | Training loop implementation showing how the auxiliary loss is computed alongside landmark regression. |

## Summary

- **PFLD outputs dual tensors**: 196-dimensional landmarks (98 points) and 3-dimensional Euler angles via the architecture in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py).
- **Input requirements**: 112×112 RGB images normalized to [0, 1], typically cropped using the bundled MTCNN detector.
- **Graph tensor names**: Retrieve landmarks via `fc:0` and pose via `pfld_fc2:0` when loading checkpoints.
- **Plug-and-play design**: Use landmarks for face alignment before feeding recognition or emotion networks, or consume pose angles directly for head tracking.
- **No external dependencies**: The pipeline relies solely on TensorFlow, NumPy, and OpenCV operations defined within the repository.

## Frequently Asked Questions

### What input image size does PFLD require?

PFLD expects 112×112 pixel RGB images with pixel values normalized to the range [0, 1]. The repository uses this fixed input dimension to maintain consistent landmark localization accuracy across different face scales.

### How do I extract usable coordinate pairs from the model output?

The `fc:0` tensor outputs a 196-dimensional vector. Reshape this to (-1, 2) to obtain 98 (x, y) coordinate pairs. Multiply these by the original face crop dimensions (width, height) to denormalize coordinates to the source image scale.

### Can I disable the pose estimation branch during inference?

Yes. While [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) defines the auxiliary branch for `euler_angles_pre`, you can choose not to fetch `pfld_fc2:0` during `sess.run()`. The landmark regression operates independently, so downstream tasks requiring only facial geometry can ignore the pose output entirely.

### Which face detector should I use with PFLD?

The repository provides MTCNN in [`mtcnn/detect_face.py`](https://github.com/guoqiangqi/pfld/blob/main/mtcnn/detect_face.py), which is tested and ready to use. However, any detector producing bounding boxes (x1, y1, x2, y2) works provided you resize crops to 112×112 before feeding them to PFLD.