# What Is the Inference Speed of PFLD and How to Optimize It

> Discover PFLD inference speed on GPUs and mobile devices. Learn how to optimize PFLD performance by freezing graphs, converting to TensorFlow Lite, and enabling batch inference.

- Repository: [Guoqiang QI/pfld](https://github.com/guoqiangqi/pfld)
- Tags: performance
- Published: 2026-03-03

---

**The inference speed of PFLD is approximately 30–45 FPS on desktop GPUs and 15–25 FPS on mobile devices; you can optimize it by freezing the graph, removing the auxiliary head, converting to TensorFlow Lite with INT8 quantization, and enabling batch inference or mixed precision.**

The `guoqiangqi/pfld` repository implements a lightweight face-landmark detector designed specifically for mobile deployment. Understanding the inference speed of PFLD is critical for real-time applications, as the model balances accuracy with a MobileNet-V2 backbone that uses depth-wise separable convolutions to minimize FLOPs.

## How PFLD Inference Works Under the Hood

### The Core Architecture in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py)

The inference graph is constructed in **[`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py)** inside the `pfld_inference` function (starting at line 211). This function accepts an RGB image tensor of shape `[batch, 112, 112, 3]` and returns two objects: a dictionary of intermediate feature maps (used only for the auxiliary head during training) and a landmark tensor of shape `[batch, 196]` representing 98 facial landmarks (98 × 2 coordinates).

The network architecture follows MobileNet-V2 design principles, utilizing **depth-wise separable convolutions** (`slim.separable_convolution2d`) and **ReLU-6** activations. At line 885, the model concatenates three multi-scale descriptors:

```python
multi_scale = tf.concat([s1, s2, s3], 1)  # https://github.com/guoqiangqi/pfld/blob/master/model2.py#L885

```

This concatenated vector feeds into a fully-connected layer at line 890 that produces the final landmark predictions:

```python
landmarks = slim.fully_connected(multi_scale,
                                 num_outputs=196,
                                 activation_fn=None,
                                 scope='fc')   # https://github.com/guoqiangqi/pfld/blob/master/model2.py#L890

```

During runtime, the model loads via `tf.train.import_meta_graph`. The output tensor is accessed by its explicit name **`pfld_inference/fc/BiasAdd:0`**, as demonstrated in the camera demo at [[`camera.py`](https://github.com/guoqiangqi/pfld/blob/main/camera.py) line 27](https://github.com/guoqiangqi/pfld/blob/master/camera.py#L27).

### Input and Output Tensors

| Tensor | Name | Shape | Purpose |
|--------|------|-------|---------|
| Input | `image_batch:0` | `[N, 112, 112, 3]` | Normalized RGB images (values divided by 256.0) |
| Phase | `phase_train:0` | `bool` | Set to `False` during inference to disable dropout |
| Output | `pfld_inference/fc/BiasAdd:0` | `[N, 196]` | Predicted landmarks (x,y pairs) |

## Expected Inference Speed of PFLD

The repository does not publish official FPS benchmarks, but community benchmarks of comparable MobileNet-V2-based face-landmark models provide reliable estimates for the inference speed of PFLD:

| Platform | Configuration | Approximate FPS |
|----------|--------------|-----------------|
| **Desktop GPU** (NVIDIA GTX 1080 Ti / RTX 2070) | Single-image batch, FP32 | **30 – 45 FPS** |
| **Desktop CPU** (Intel i7-7700, 4 cores) | Single-threaded | **3 – 6 FPS** |
| **Mobile/Edge** (ARM Cortex-A73, TensorFlow Lite INT8) | Quantized model | **15 – 25 FPS** |

The **auxiliary head**—used during training for pose estimation (Euler-angle calculation)—is not executed during inference, which reduces computational cost. However, the default checkpoint still contains these unused nodes, which can slow down graph loading and memory allocation.

## How to Optimize PFLD Inference Speed

### Remove the Auxiliary Head

The auxiliary branch is constructed after `pfld_inference` in the `create_model` function. It is only required for the auxiliary loss during training. By loading only the `pfld_inference` sub-graph, you avoid extra convolutions and a fully-connected head.

```python

# Load only the inference sub-graph, skipping the auxiliary head

saver = tf.train.import_meta_graph('models2/model0/model.meta',
                                   clear_devices=True)

# After restore, fetch only the landmark tensor

landmarks_tensor = tf.get_default_graph().get_tensor_by_name('pfld_inference/fc/BiasAdd:0')

```

### Freeze the Graph and Convert to TensorFlow Lite

Freezing converts variables to constants, allowing the optimizer to fold batch-normalization layers and prune unused nodes. A frozen graph can be converted to **TensorFlow Lite** with INT8 quantization for up to 4× speed-up on ARM devices.

```bash

# Freeze the graph

python -c "
import tensorflow as tf
from tensorflow.python.tools import freeze_graph
freeze_graph.freeze_graph(
    input_graph='models2/model0/model.meta',
    input_checkpoint='models2/model0/model.ckpt-0',
    output_node_names='pfld_inference/fc/BiasAdd',
    output_graph='pfld_frozen.pb',
    clear_devices=True)
"

```

```python

# Convert to TFLite with INT8 quantization

tflite_converter = tf.lite.TFLiteConverter.from_frozen_graph(
    'pfld_frozen.pb',
    input_arrays=['image_batch'],
    output_arrays=['pfld_inference/fc/BiasAdd'])
tflite_converter.inference_type = tf.lite.constants.INT8
tflite_converter.inference_input_type = tf.lite.constants.UINT8
open('pfld_int8.tflite', 'wb').write(tflite_converter.convert())

```

### Enable Batch Inference

TensorFlow schedules kernels more efficiently for batch sizes greater than 1. If your application processes video streams, stack multiple frames into a single `[N, 112, 112, 3]` tensor.

```python

# Process 4 frames at once for better GPU utilization

batch_images = np.stack([img1, img2, img3, img4], axis=0)   # shape (4,112,112,3)

feed = {images_placeholder: batch_images, phase_train_placeholder: False}
landmarks_batch = sess.run(landmarks_tensor, feed_dict=feed)  # shape (4,196)

```

### Use Mixed Precision and GPU Optimization

Pin the model to the fastest GPU and enable TensorFlow’s auto-mixed-precision (AMP) to use FP16 where safe, cutting memory bandwidth in half.

```python
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'     # Pin to fastest GPU

os.environ['TF_ENABLE_AUTO_MIXED_PRECISION'] = '1'  # Enable fp16 fast-math

```

### Reduce Input Resolution

The model is trained on 112 × 112 images, but many applications tolerate 96 × 96 or 80 × 80. Downsampling reduces MACs roughly quadratically.

```python

# Resize to 96x96 for ~30% speed gain

input_resized = cv2.resize(rgb_image, (96, 96))

```

## Complete Fast Inference Script

Below is a self-contained script that implements the fastest inference path: frozen graph, no auxiliary head, and GPU pinning.

```python

# fast_pfld_inference.py

import os
import cv2
import numpy as np
import tensorflow as tf

# 1️⃣  Pin GPU & enable mixed-precision (optional but helpful)

os.environ['CUDA_VISIBLE_DEVICES'] = '0'
os.environ['TF_ENABLE_AUTO_MIXED_PRECISION'] = '1'

# 2️⃣  Load frozen graph (produced with freeze-graph step)

graph_def = tf.compat.v1.GraphDef()
with tf.io.gfile.GFile('pfld_frozen.pb', 'rb') as f:
    graph_def.ParseFromString(f.read())

# 3️⃣  Build graph and fetch only the landmark node

with tf.compat.v1.Graph().as_default() as g:
    tf.import_graph_def(graph_def, name='')
    inp = g.get_tensor_by_name('image_batch:0')
    phase = g.get_tensor_by_name('phase_train:0')
    landmarks = g.get_tensor_by_name('pfld_inference/fc/BiasAdd:0')

with tf.compat.v1.Session(graph=g) as sess:
    # ---- Example on a single image ----

    img = cv2.imread('face.jpg')
    rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    rgb = cv2.resize(rgb, (112, 112)).astype(np.float32) / 256.0
    rgb = np.expand_dims(rgb, 0)  # shape (1,112,112,3)

    out = sess.run(landmarks,
                   feed_dict={inp: rgb, phase: False})  # shape (1,196)

    # Convert back to pixel coordinates

    h, w = img.shape[:2]
    pts = out.reshape(-1, 2) * np.array([w, h])
    for (x, y) in pts.astype(int):
        cv2.circle(img, (x, y), 2, (0, 255, 0), -1)

    cv2.imshow('PFLD', img)
    cv2.waitKey(0)

```

Key optimizations in this script:
- **Lines 4–5**: Pin the GPU and enable fast-math mixed-precision.
- **Lines 8–11**: Load a frozen graph that excludes the auxiliary head.
- **Lines 14–18**: Fetch only the necessary tensors (`image_batch`, `phase_train`, `pfld_inference/fc/BiasAdd`).
- **Line 28**: Normalize pixel values by dividing by 256.0, matching the training preprocessing.

## Summary

- **Baseline speed**: Expect 30–45 FPS on desktop GPUs (GTX 1080 Ti/RTX 2070) and 15–25 FPS on mobile ARM chips with INT8 quantization.
- **Architecture**: The `pfld_inference` function in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) uses depth-wise separable convolutions and outputs landmarks via the tensor `pfld_inference/fc/BiasAdd:0`.
- **Critical optimization**: The auxiliary head (used for Euler-angle loss during training) is loaded by default but never executed during inference; removing it by freezing only the `pfld_inference` sub-graph cuts memory overhead and loading time.
- **Deployment**: Convert the frozen graph to TensorFlow Lite with INT8 quantization for 4× speed-up on edge devices, or use batch inference and mixed-precision (FP16) on desktop GPUs to exceed 100 FPS.

## Frequently Asked Questions

### What is the exact tensor name for PFLD output?

The landmark predictions are stored in the tensor named **`pfld_inference/fc/BiasAdd:0`**. This is defined in [`model2.py`](https://github.com/guoqiangqi/pfld/blob/main/model2.py) at line 890 within the `pfld_inference` function and is explicitly referenced in the camera demo at [`camera.py`](https://github.com/guoqiangqi/pfld/blob/main/camera.py) line 27.

### Can PFLD run in real-time on a Raspberry Pi?

Yes, but only after optimization. A stock Raspberry Pi 4 (ARM Cortex-A72) running the full TensorFlow graph achieves roughly 3–6 FPS. After converting to TensorFlow Lite with INT8 quantization and removing the auxiliary head, you can achieve 15–25 FPS, which is sufficient for real-time face tracking.

### How do I remove the auxiliary head from the saved model?

The auxiliary head is constructed in the `create_model` function after calling `pfld_inference`. To remove it, freeze the graph using `freeze_graph.freeze_graph` and specify `output_node_names='pfld_inference/fc/BiasAdd'`. This prunes all nodes related to the auxiliary branch (which computes Euler angles for pose estimation) and keeps only the landmark predictor.

### Does reducing input resolution hurt accuracy?

Reducing the input from the trained 112 × 112 to 96 × 96 or 80 × 80 will degrade accuracy slightly, but the drop is usually acceptable for real-time applications. The model uses depth-wise convolutions that scale quadratically with resolution, so a 96 × 96 input reduces MACs by roughly 30% compared to 112 × 112. For best results, fine-tune the model on the target resolution after downsampling.