How to Process Partially Occluded or Incomplete Faces with face_recognition

The face_recognition library automatically detects and processes partially occluded faces by clipping bounding boxes to image edges, allowing you to extract encodings even from incomplete facial regions.

The face_recognition library leverages dlib's HOG and CNN detectors to locate faces in images, even when those faces extend beyond the frame or are partially covered. Understanding how to process partially occluded or incomplete faces is essential for robust face recognition systems that handle real-world photography where perfect framing isn't always possible.

How face_recognition Handles Partial Face Detection

The library uses dlib's pre-trained models to scan images for facial features. When the detector identifies a face region that extends beyond the image boundaries, the internal _trim_css_to_bounds function in face_recognition/api.py (lines 52-60) automatically clips the coordinates to valid image dimensions.

This clipping mechanism ensures that partially visible faces—such as those cut off by the camera frame—are reported with truncated bounding boxes rather than being discarded entirely. The _raw_face_locations function (lines 108-122) serves as the entry point, routing requests to either the HOG (face_detector) or CNN (cnn_face_detector) model based on your configuration.

Detecting Partially Occluded Faces in Practice

Loading and Detecting Partial Faces

To detect faces that are partially outside the frame, use the standard face_locations function. The library includes test cases with obama_partial_face.jpg and obama_partial_face2.jpg that verify this behavior in tests/test_face_recognition.py (lines 81-92).

import face_recognition
from PIL import Image

# Load an image with a partially visible face

image = face_recognition.load_image_file("obama_partial_face.jpg")

# Detect faces using the default HOG model

face_locations = face_recognition.face_locations(image)

print(f"Detected {len(face_locations)} face(s)")

# Output: Detected 1 face(s)

# Crop and display the partial face

for top, right, bottom, left in face_locations:
    face_image = image[top:bottom, left:right]
    pil_image = Image.fromarray(face_image)
    pil_image.show()

Handling Edge Cases with Upsampling

For small or heavily clipped faces, increase the number_of_times_to_upsample parameter. This resizes the image internally before detection, improving the chances of detecting partial facial features.

# Upsample twice for better detection of small/partial faces

face_locations = face_recognition.face_locations(
    image, 
    number_of_times_to_upsample=2
)

Generating Face Encodings from Incomplete Faces

Once you have the clipped bounding box, you can generate a 128-dimensional face encoding using face_encodings. Note that the encoding derives from only the visible portion of the face, which may reduce matching accuracy compared to full-face encodings.

import face_recognition

image = face_recognition.load_image_file("obama_partial_face2.jpg")
boxes = face_recognition.face_locations(image)

# Generate encoding from the partial face region

encoding = face_recognition.face_encodings(image, known_face_locations=boxes)[0]

# Compare with a known encoding (example)

known_encoding = [...]  # Your reference encoding

matches = face_recognition.compare_faces([known_encoding], encoding)
distance = face_recognition.face_distance([known_encoding], encoding)

print(f"Match: {matches[0]}, Distance: {distance[0]}")

Using the CNN Model for Better Accuracy

If you have GPU support, the CNN model often provides more accurate bounding boxes for partial faces, though at the cost of processing speed.

# Use CNN model for potentially better partial face detection

face_locations = face_recognition.face_locations(image, model="cnn")

Key Implementation Details

The robust handling of partial faces relies on several internal functions in face_recognition/api.py:

• _raw_face_locations (lines 108-122): Routes detection to HOG or CNN models and returns dlib rectangle objects.
• _rect_to_css (lines 32-40): Converts dlib's (top, right, bottom, left) format to the CSS-style tuple used throughout the API.
• _trim_css_to_bounds (lines 52-60): Clips coordinates to image dimensions, ensuring partial faces return valid, truncated bounding boxes rather than being discarded.

The test suite in tests/test_face_recognition.py (lines 81-92) explicitly validates this behavior using obama_partial_face.jpg and obama_partial_face2.jpg, confirming that single bounding boxes are returned for faces extending beyond frame edges.

Summary

• Automatic clipping: The _trim_css_to_bounds function ensures partially occluded faces are reported with truncated coordinates rather than ignored.
• Safe cropping: Returned bounding boxes are guaranteed to be within image dimensions, preventing index errors when slicing NumPy arrays.
• Flexible detection: Both HOG (CPU) and CNN (GPU) models support partial face detection, with optional upsampling for small fragments.
• Encoding limitations: Face encodings generated from partial regions rely only on visible features, which may affect matching accuracy compared to full-face templates.

Frequently Asked Questions

Can face_recognition detect faces that are half outside the frame?

Yes. The library automatically clips bounding boxes to image edges using the internal _trim_css_to_bounds function in face_recognition/api.py. This means faces extending beyond the frame boundaries are detected and returned with truncated coordinates rather than being discarded.

Does partial occlusion affect face encoding accuracy?

Yes. When you call face_encodings on a partially visible face, the 128-dimensional vector is computed only from the visible facial region. This reduces the amount of discriminative information available, potentially increasing false positive rates when comparing against full-face templates using compare_faces or face_distance.

Should I use HOG or CNN for partially visible faces?

Both models support partial face detection. The default HOG model is faster on CPU and works well for most partial face scenarios. The CNN model (enabled with model="cnn") may provide more accurate bounding boxes for challenging partial occlusions but requires GPU acceleration for practical performance.

How does the library prevent index errors when cropping partial faces?

The _trim_css_to_bounds function (lines 52-60 in face_recognition/api.py) ensures all returned coordinates stay within valid image dimensions. When you slice the NumPy array using image[top:bottom, left:right], the indices are guaranteed to be valid, preventing IndexError exceptions even when the face extends beyond the original frame.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}