How PFLD Calculates Euler Angles for Head Pose Estimation: A Complete Guide

PFLD computes head pose Euler angles by solving a Perspective-n-Point (PnP) problem that maps 3D facial model points to 2D image landmarks, then decomposes the rotation matrix into pitch, yaw, and roll using OpenCV.

The PFLD (Pose-Free Landmark Detection) implementation in the guoqiangqi/pfld repository provides a robust pipeline for extracting head orientation from facial images. Understanding how PFLD handles Euler angle calculations requires examining the geometric transformation pipeline that bridges 3D facial geometry with 2D image coordinates.

The Mathematical Foundation of PFLD Euler Angle Calculations

3D Facial Model Template Construction

PFLD defines a canonical 3D facial template consisting of 14 anthropometric landmarks that represent key facial features including eyebrows, eyes, nose bridge, mouth corners, and chin. These points are expressed in a head-centered coordinate system where the origin typically aligns with the face center.

In euler_angles_utils.py (lines 45-58) and euler_angles.py (lines 158-186), the 3D template is hardcoded as a NumPy array representing the spatial coordinates of these tracked points. This template serves as the reference model that PnP algorithms attempt to align with observed 2D landmarks.

Camera Intrinsic Matrix Calibration

Before solving the pose estimation problem, PFLD constructs a camera intrinsic matrix based on assumed image resolution and field of view. The implementation assumes a 60-degree horizontal field of view and calculates focal length accordingly.

In euler_angles_utils.py (lines 9-18) and euler_angles.py (lines 35-43), the camera matrix is built as follows:

c_x = cam_w / 2
c_y = cam_h / 2
f_x = c_x / np.tan(60/2 * np.pi / 180)
f_y = f_x
camera_matrix = np.float32([[f_x, 0, c_x],
                           [0, f_y, c_y],
                           [0,   0,   1]])

This matrix maps 3D camera coordinates to 2D image coordinates, accounting for focal length and principal point offset.

Step-by-Step Implementation in PFLD

Step 1: Landmark Detection and Point Correspondence

The Euler angle calculation begins with establishing correspondences between the 3D facial template and detected 2D landmarks. In euler_angles.py (lines 70-78), the PnpHeadPoseEstimator._return_landmarks method uses dlib's 68-point shape predictor to extract the specific subset of landmarks defined in TRACKED_POINTS.

The calculate_pitch_yaw_roll function in euler_angles_utils.py expects a pre-computed landmarks_2D array containing the 14 tracked points in the same order as the 3D template.

Step 2: Solving the PnP Problem with OpenCV

With correspondences established, PFLD solves the Perspective-n-Point problem using OpenCV's solvePnP function. This step determines the rotation and translation vectors that align the 3D model with the 2D observations.

In euler_angles_utils.py (lines 70-73) and euler_angles.py (lines 202-206), the implementation calls:

retval, rvec, tvec = cv2.solvePnP(landmarks_3D,
                                 landmarks_2D,
                                 camera_matrix,
                                 camera_distortion)

The function returns a rotation vector (rvec) and translation vector (tvec) representing the head pose in camera coordinates.

Step 3: Rotation Vector to Matrix Conversion

Before extracting Euler angles, the rotation vector must be converted to a rotation matrix using Rodrigues' formula. In euler_angles_utils.py (lines 77-78) and euler_angles.py (lines 208-209), PFLD uses OpenCV's Rodrigues function:

rmat, _ = cv2.Rodrigues(rvec)

This produces a 3×3 rotation matrix (rmat) that describes the orientation of the head relative to the camera frame.

Step 4: Extracting Pitch, Yaw, and Roll Angles

The final step converts the rotation matrix into Euler angles (pitch, yaw, roll). PFLD implements two methods for this conversion:

Primary Method: decomposeProjectionMatrix

In euler_angles_utils.py (lines 80-84) and euler_angles.py (lines 211-214), the code constructs a projection matrix and uses OpenCV's decomposition function:

pose_mat = cv2.hconcat((rmat, tvec))
_, _, _, _, _, _, euler_angles = cv2.decomposeProjectionMatrix(pose_mat)
pitch, yaw, roll = map(lambda a: a[0], euler_angles)

Fallback Method: Manual Conversion

As a fallback, euler_angles.py (lines 226-246) provides rotationMatrixToEulerAngles, which manually extracts angles following the X-Y-Z (roll-pitch-yaw) convention and handles gimbal-lock singularities.

By default, angles are returned in degrees, though the utilities support returning radians when radians=True.

Code Examples: Using PFLD's Euler Angle Utilities

Direct Calculation with euler_angles_utils.py

For applications that already have 2D landmark coordinates, the standalone utility function provides the most direct path to Euler angles:

import cv2
import numpy as np
from euler_angles_utils import calculate_pitch_yaw_roll

# landmarks_2D must contain the 14 tracked points in the specific order

# defined by the 3D template (eyebrows, eyes, nose, mouth, chin)

landmarks_2D = np.array([
    [x1, y1],  # left eyebrow outer

    [x2, y2],  # left eyebrow inner

    # ... remaining 12 points

], dtype=np.float32)

pitch, yaw, roll = calculate_pitch_yaw_roll(
    landmarks_2D,
    cam_w=640,
    cam_h=480,
    radians=False
)

print(f'Pitch: {pitch:.2f}°, Yaw: {yaw:.2f}°, Roll: {roll:.2f}°')

This approach bypasses face detection and assumes the caller has already extracted the specific 14-point subset that matches PFLD's 3D model template.

Full Pipeline with PnpHeadPoseEstimator

For end-to-end head pose estimation from raw images, the PnpHeadPoseEstimator class integrates dlib face detection with the Euler angle calculation:

import cv2
from euler_angles import PnpHeadPoseEstimator

# Initialize with dlib's 68-point shape predictor

shape_predictor_path = './shape_predictor_68_face_landmarks.dat'
estimator = PnpHeadPoseEstimator(
    shape_predictor_path,
    cam_w=640,
    cam_h=480
)

cap = cv2.VideoCapture(0)
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Returns list of three 1-element arrays: [[pitch], [yaw], [roll]]

    euler = estimator.return_pitch_yaw_roll(frame)
    
    if euler:
        pitch, yaw, roll = map(lambda a: a[0], euler)
        print(f'Pitch: {pitch:.2f}, Yaw: {yaw:.2f}, Roll: {roll:.2f}')
        
        # Visualization code here...

        
    cv2.imshow('Head pose', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

The class handles the complete workflow: face detection, landmark extraction from the 68-point dlib output, selection of the 14 tracked points, camera matrix construction, PnP solving, and final Euler angle decomposition.

Summary

• PFLD calculates Euler angles by solving a Perspective-n-Point problem that aligns a predefined 3D facial template with detected 2D landmarks.
• Core implementation resides in euler_angles_utils.py (standalone functions) and euler_angles.py (full estimator class).
• Camera calibration assumes a 60° horizontal field of view and constructs the intrinsic matrix from image dimensions.
• OpenCV pipeline uses solvePnP for pose estimation, Rodrigues for rotation vector conversion, and decomposeProjectionMatrix for Euler angle extraction.
• Fallback method manually converts rotation matrices to Euler angles with gimbal-lock protection when OpenCV decomposition is unavailable.

Frequently Asked Questions

What is the PnP problem in the context of PFLD?

The Perspective-n-Point (PnP) problem involves estimating the pose (rotation and translation) of a calibrated camera relative to a known 3D object, given n 3D points and their corresponding 2D projections. In PFLD, the "object" is a canonical 3D face model with 14 anthropometric points, and the 2D projections are the detected facial landmarks. OpenCV's solvePnP function computes the rotation vector (rvec) and translation vector (tvec) that minimize the reprojection error between the 3D model and 2D observations.

Why does PFLD use 14 specific facial landmarks for pose estimation?

PFLD selects 14 landmarks from the full 68-point dlib facial shape predictor to balance geometric stability with computational efficiency. These points correspond to distinct facial features (eyebrow corners, eye corners, nose tip, mouth corners, chin) that provide sufficient 3D geometric constraints to solve the PnP problem while excluding less stable points like cheek contours. The specific selection is defined in the TRACKED_POINTS array within euler_angles.py (lines 70-78), ensuring consistent correspondence between the 3D template and detected 2D landmarks.

How accurate are the Euler angles calculated by PFLD?

The accuracy of PFLD's Euler angles depends on the precision of the underlying 2D landmark detection and the validity of the camera calibration assumptions. The implementation assumes a 60-degree horizontal field of view and constructs the camera matrix from image dimensions, which introduces error if the actual camera parameters differ. The PnP solution with solvePnP typically achieves sub-degree accuracy under ideal conditions, but real-world performance varies with face detection stability, occlusion, and extreme head poses. The fallback manual Euler extraction in rotationMatrixToEulerAngles includes gimbal-lock handling to maintain stability when the head approaches vertical alignment.

Can PFLD return Euler angles in radians instead of degrees?

Yes, both the standalone calculate_pitch_yaw_roll function in euler_angles_utils.py and the PnpHeadPoseEstimator class in euler_angles.py support returning angles in radians. By passing radians=True to these functions, the implementation skips the degree conversion step and returns the raw values from the rotation matrix decomposition. This is useful for applications that require radian inputs for further trigonometric calculations or when integrating with robotics frameworks that use SI units. The default behavior returns degrees for human-readable output and debugging purposes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}