# PFLD Model Architecture: MobileNet-V2 Backbone, Multi-Scale Features, and Landmark Regression

> Explore the PFLD model architecture featuring MobileNet-V2 backbone multi-scale features and landmark regression. Discover its components and how it predicts 98 facial landmarks and 3D head pose.

- Repository: [Guoqiang QI/pfld](https://github.com/guoqiangqi/pfld)
- Tags: deep-dive
- Published: 2026-03-02

---

**The PFLD (Progressive Face Landmark Detection) model architecture consists of a MobileNet-V2-style backbone for hierarchical feature extraction, a multi-scale inference head that aggregates pooled representations to predict 98 facial landmarks (196 values), and an auxiliary sub-network that estimates 3D head pose (Euler angles) to improve accuracy, all implemented in TensorFlow Slim within the `guoqiangqi/pfld` repository.**

The PFLD model architecture implemented in the `guoqiangqi/pfld` repository delivers a lightweight, real-time facial landmark detection system optimized for mobile deployment. Built entirely with TensorFlow Slim (`tf.contrib.slim`) operations, this compact convolutional neural network processes 112×112 RGB face images through a hierarchical feature pyramid to regress precise facial keypoints. The complete implementation resides in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py), where the `create_model` function wires together the backbone extractor, landmark regression head, and auxiliary pose estimation network.

## PFLD Model Architecture Overview

The architecture follows a three-stage design pattern common in efficient face analysis systems:

1.  **Backbone Feature Extractor** – A MobileNet-V2-style network with depthwise-separable convolutions and residual connections that generates multi-scale feature maps.
2.  **PFLD Inference Head** – A multi-scale aggregation module that pools features at different resolutions and regresses 98 facial landmarks through a fully-connected layer.
3.  **Auxiliary Pose Head** – A lightweight convolutional sub-network that processes intermediate features to predict yaw, pitch, and roll angles.

All components share batch normalization parameters (decay 0.995, epsilon 0.001) and utilize global average pooling to minimize parameter count while maintaining spatial accuracy.

## Backbone – MobileNet-V2-Style Feature Extractor

The `mobilenet_v2` function in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) constructs the lightweight feature pyramid through progressive downsampling and residual feature reuse.

**Key architectural stages:**

-   **Initial Convolution** – A 3×3 convolution with stride 2 and 32 filters reduces the spatial dimensions from 112×112 to 56×56 (lines 24–27).
-   **Depthwise-Separable Blocks** – Repeated blocks (`conv2_1` through `conv5_4`) apply depthwise 3×3 convolutions followed by pointwise projections to extract features efficiently (lines 28–34).
-   **Residual Connections** – Each major block adds the output of the last depthwise convolution to the block's input, enabling gradient flow without inflating parameters. For example, `block_3_2 = conv3_1 + conv3_2` (lines 53–56).
-   **Feature Map Collection** – The backbone populates a `features` dictionary storing intermediate tensors (`feature2` through `feature6`) at different resolutions for later multi-scale fusion (lines 34–58).
-   **Deep Representation** – The final `conv7_4` layer produces a 320-channel tensor at 7×7 resolution, serving as the deepest feature representation for downstream heads (lines 200–208).

## PFLD Inference Head

Implemented in the `pfld_inference` function (lines 24–91), this head transforms backbone features into landmark coordinates through progressive refinement and multi-scale pooling.

### Multi-Scale Feature Aggregation

After processing through residual blocks (`conv3_*`, `conv4_*`, `conv5_*`), the network extracts representations at three distinct scales:

```python
avg_pool1 = slim.avg_pool2d(conv6_1,
    [conv6_1.get_shape()[1], conv6_1.get_shape()[2]], stride=1)
avg_pool2 = slim.avg_pool2d(conv7,
    [conv7.get_shape()[1], conv7.get_shape()[2]], stride=1)
s1 = slim.flatten(avg_pool1)
s2 = slim.flatten(avg_pool2)
s3 = slim.flatten(conv8)          # 1×1×128 global conv

multi_scale = tf.concat([s1, s2, s3], 1)   # lines 88-90

```

This concatenation combines global context from `conv6_1`, high-level semantics from `conv7`, and compressed features from the final 1×1 convolution (`conv8`), creating a rich 896-dimensional representation (exact size depends on channel dimensions) that captures both fine-grained details and holistic face structure.

### Landmark Regression Layer

The aggregated features feed into a single fully-connected layer:

```python
landmarks = slim.fully_connected(
    multi_scale, num_outputs=196, activation_fn=None, scope='fc')

```

With **196 output values**, this layer predicts 98 (x, y) coordinate pairs representing facial landmarks. The linear activation (no non-linearity) allows unrestricted regression of continuous coordinate values.

## Auxiliary Head for Pose Estimation

The auxiliary network processes `features['auxiliary_input']` (specifically the `block3_5` output at 28×28 resolution) to provide geometric constraints during training.

**Layer configuration (lines 18–35 in `create_model`):**

| Layer | Configuration | Activation |
|-------|--------------|------------|
| Conv 1 | 128 filters, 3×3, stride 2 | ReLU |
| Conv 2 | 128 filters, 3×3, stride 1 | ReLU |
| Conv 3 | 32 filters, 3×3, stride 2 | ReLU |
| Conv 4 | 128 filters, 7×7, stride 1 | ReLU |
| Max-Pool | 3×3, stride 1, padding SAME | – |
| FC 1 | 32 units | Linear |
| FC 2 (Output) | 3 units (yaw, pitch, roll) | Linear |

This sub-network outputs `euler_angles_pre`, a 3-dimensional vector representing head orientation that can be incorporated into the total loss function to improve landmark robustness under extreme poses.

## Model Integration in create_model

The `create_model` function (lines 93–98) orchestrates the complete PFLD model architecture:

1.  Configures **batch normalization** parameters with decay 0.995 and epsilon 0.001.
2.  Invokes `pfld_inference` to generate the **feature dictionary** and **landmark predictions** (`landmarks_pre`).
3.  Computes the **mean squared error (MSE)** between predictions and ground truth landmarks.
4.  Executes the **auxiliary head** on `features['auxiliary_input']` to produce pose estimates (`euler_angles_pre`).
5.  Returns the prediction tensors and loss value for the training loop.

## Implementation Example: Building the PFLD Graph

The following script demonstrates how to instantiate the complete architecture using the repository's API:

```python
import tensorflow as tf
from model2 import create_model, LandmarkImage

# 1. Define input placeholders

img_ph = tf.placeholder(tf.float32, [None, 112, 112, 3], name='input_image')
lm_ph  = tf.placeholder(tf.float32, [None, 196], name='ground_truth')
is_train = tf.placeholder(tf.bool, name='is_training')

# 2. Configure model arguments

class Args:
    weight_decay = 5e-4
args = Args()

# 3. Construct the PFLD model architecture

landmarks_pred, loss, pose_pred = create_model(
    input=img_ph,
    landmark=lm_ph,
    phase_train=is_train,
    args=args)

# 4. Setup optimizer

optimizer = tf.train.AdamOptimizer(learning_rate=1e-3)
train_op = optimizer.minimize(loss)

# 5. Execute training step

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    _, cur_loss, cur_landmarks, cur_pose = sess.run(
        [train_op, loss, landmarks_pred, pose_pred],
        feed_dict={img_ph: batch_imgs,
                   lm_ph: batch_landmarks,
                   is_train: True})
    print('Landmark Loss:', cur_loss)

```

This implementation creates placeholders matching the expected 112×112 input resolution, instantiates the Args configuration container used by [`train_model.py`](https://github.com/guoqiangqi/pfld/blob/main/train_model.py), and builds the full computation graph including both the landmark regression and auxiliary pose estimation pathways.

## Summary

-   **Backbone Design:** The PFLD model architecture utilizes a MobileNet-V2-style feature extractor with depthwise-separable convolutions and residual connections, defined in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) within the `mobilenet_v2` function.
-   **Multi-Scale Aggregation:** The inference head concatenates globally pooled features from three different network depths (`conv6_1`, `conv7`, and `conv8`) to capture both local details and global face structure before regression.
-   **Output Representation:** The model predicts **98 facial landmarks** (196 coordinate values) through a single fully-connected layer and **3 Euler angles** (yaw, pitch, roll) through an auxiliary convolutional head.
-   **Implementation Framework:** Built entirely with TensorFlow Slim operations, the architecture supports 112×112 RGB inputs and employs batch normalization throughout for training stability.

## Frequently Asked Questions

### What input dimensions does the PFLD model architecture require?

The PFLD model architecture expects input tensors of shape **[None, 112, 112, 3]** representing batches of 112×112 pixel RGB face images. This resolution is hardcoded in the placeholder definitions within [`train_model.py`](https://github.com/guoqiangqi/pfld/blob/main/train_model.py) and processed through the initial 3×3 convolution with stride 2 in the `mobilenet_v2` backbone function.

### How many facial landmarks does the PFLD architecture predict?

The architecture predicts **98 facial landmarks**, outputting 196 continuous values (98 x-coordinate and 98 y-coordinate pairs) from the final fully-connected layer in the `pfld_inference` function. This dense configuration provides detailed coverage of facial contours, eyes, nose, and mouth boundaries.

### What is the purpose of the auxiliary head in the PFLD model?

The auxiliary head predicts **3D head pose** (yaw, pitch, and roll Euler angles) by processing intermediate features from the backbone's `block3_5` layer. According to the source code in `create_model`, this auxiliary task provides geometric constraints during training that improve the main landmark detector's robustness to extreme head poses and partial occlusions.

### Which deep learning framework does the PFLD implementation use?

The implementation uses **TensorFlow 1.x** with the **TensorFlow Slim** (`tf.contrib.slim`) high-level API for layer construction. All convolutional, pooling, and fully-connected operations are Slim wrappers, and the model defines explicit batch normalization parameters with decay 0.995 and epsilon 0.001 for consistent training behavior.