How to Fine-Tune a Pre-Trained PFLD Model on Custom Facial Landmark Data

Fine-tuning a pre-trained PFLD model involves converting your dataset to NumPy format with data/SetPreparation.py, loading pre-trained weights in model2.py, adjusting hyperparameters in train_model.py (typically lowering the learning rate to 1e-5 and freezing early backbone layers), and running the training loop before validating with test_model.py and deploying via camera.py.

The PFLD (Progressive Face Localization and Detection) repository provides a lightweight facial-landmark detector optimized for real-time applications. This guide explains how to fine-tune a pre-trained PFLD model on your own dataset to improve accuracy on domain-specific imagery while retaining the model's efficient MobileNet-style architecture.

Prepare Your Training Dataset

Convert Raw Annotations to NumPy Format

The training pipeline expects face images and their corresponding 68-point (or 98-point) landmark annotations in NumPy format. Use the helper script data/SetPreparation.py to convert raw data into the expected format.

python data/SetPreparation.py --input_dir data/raw_images \
                              --anno_file data/annotations.txt \
                              --output_dir data/prepared

Optional Data Augmentation

For dataset compression or augmentation, utilize utilities in the tools/ directory. The tools/frame_cut.py script handles frame extraction, while tools/emotion_compress.py manages compression tasks.

Configure the Model Architecture

Load Pre-Trained Weights

The core network implementation resides in model2.py. This file defines a MobileNet-style backbone followed by depth-wise separable convolutions that output landmark coordinates. It contains the logic to load pre-trained weights trained on the original 300-W dataset, serving as the foundation for fine-tuning.

Configure Fine-Tuning Parameters

Adjust Hyperparameters for Transfer Learning

Hyperparameters including learning rate, batch size, number of epochs, and loss weights are defined in train_model.py. For effective fine-tuning:

• Lower the learning rate from the default 1e-4 to 1e-5 to prevent catastrophic forgetting
• Freeze early backbone layers to preserve generic feature representations
• Allow later layers to adapt to new data distributions

Execute the PFLD Fine-Tuning Process

The train_model.py script orchestrates the complete training workflow:

• Loads data from the NumPy files generated in the preparation step
• Constructs the model architecture
• Computes losses using auxiliary pose regularization terms from euler_angles_utils.py
• Updates optimizer states
• Logs progress to data/log0.txt
• Saves periodic checkpoints to the specified directory

python train_model.py \
    --train_data data/prepared/train.npy \
    --val_data   data/prepared/val.npy \
    --pretrained model2.py:pretrained_weights.pth \
    --batch_size 64 \
    --epochs 100 \
    --learning_rate 1e-5 \
    --freeze_backbone True \
    --log_file data/log0.txt \
    --checkpoint_dir checkpoints/

Validate and Deploy the Fine-Tuned Model

Evaluate Model Performance

Run test_model.py to assess accuracy on held-out validation data:

python test_model.py \
    --model checkpoints/best_finetuned.pth \
    --test_data data/prepared/test.npy

Real-Time Inference

Export the fine-tuned weights (saved as .pth files) and integrate them into the inference pipeline. Replace the original checkpoint in camera.py to observe updated facial-landmark predictions in real time:

python camera.py --model checkpoints/best_finetuned.pth

Summary

• Data Preparation: Convert raw images and 68-point or 98-point annotations to NumPy format using data/SetPreparation.py
• Model Initialization: Load pre-trained weights via model2.py to leverage 300-W dataset knowledge
• Fine-Tuning Strategy: Reduce learning rate to 1e-5 and freeze backbone layers in train_model.py to preserve generic features
• Training Execution: Run train_model.py with pose regularization from euler_angles_utils.py and monitor logs in data/log0.txt
• Deployment: Validate with test_model.py and deploy via camera.py for real-time facial landmark detection

Frequently Asked Questions

What is the expected input format for fine-tuning a PFLD model?

The training pipeline expects NumPy arrays containing face images paired with 68-point or 98-point landmark coordinates. Use data/SetPreparation.py to convert raw images and annotation files into the required format, specifying input directories and annotation file paths via command-line arguments.

How do I prevent overfitting when fine-tuning on a small dataset?

Freeze the early layers of the MobileNet-style backbone by setting --freeze_backbone True in train_model.py, which preserves generic low-level feature extractors. Additionally, lower the learning rate to 1e-5 and utilize data augmentation tools like tools/frame_cut.py and tools/emotion_compress.py to artificially expand your training set.

Where does the PFLD model store training checkpoints and logs?

During training, train_model.py writes progress logs to data/log0.txt and saves model checkpoints to the directory specified by --checkpoint_dir. The best model weights are stored as .pth files, which can be directly loaded into test_model.py for evaluation or camera.py for real-time inference.

Can I fine-tune the PFLD model for a different number of landmarks?

While the pre-trained weights in model2.py are optimized for 68-point or 98-point configurations, you can modify the final output layer in the architecture definition to match your specific landmark count. However, this requires adjusting the model structure before loading pre-trained weights and retraining the modified layers from scratch while potentially freezing the backbone.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}