How to Train an SVM Model for Multi-Face Recognition with face_recognition

Train an SVM classifier on 128-dimensional face encodings generated by the face_recognition library to identify multiple individuals in a single image with high accuracy and fast inference.

The face_recognition library by ageitgey provides a straightforward Python API for face detection and recognition using dlib's deep learning models. When you need to identify multiple people in photographs, you can train an SVM model for multi-face recognition using the 128-dimensional face embeddings that the library extracts from each detected face.

Understanding Face Embeddings and SVM Classification

The face_recognition library converts each detected face into a 128-dimensional embedding using dlib's deep learning model. These embeddings are deterministic: the same face always produces a very similar vector, while different faces produce vectors that are far apart in Euclidean space.

An SVM (Support Vector Classifier) from scikit-learn can be trained on those embeddings to perform multi-face identification. The classifier learns a decision boundary that separates each person's embedding cluster, making it robust for small datasets and fast at inference time.

Preparing Your Training Dataset

Organize your training images in a directory tree where each sub-folder's name is the label (person's name) and contains one or more portrait images of that person.

train_dir/
    alice/
        img1.jpg
        img2.jpg
    bob/
        img1.jpg
        img2.jpg
    …

Only use images that contain exactly one face. The training script will skip any images with zero or multiple faces to avoid ambiguous labels.

Loading and Encoding Training Faces

Use the face_recognition API to load images, detect faces, and extract 128-dimensional encodings. The heavy lifting is done in face_recognition/api.py, specifically in the face_encodings function, which calls face_encoder.compute_face_descriptor.

import os
import face_recognition
from sklearn import svm

# Prepare training data

train_root = "train_dir"
encodings = []
names = []

for person_name in os.listdir(train_root):
    person_path = os.path.join(train_root, person_name)
    if not os.path.isdir(person_path):
        continue

    for img_name in os.listdir(person_path):
        img_path = os.path.join(person_path, img_name)
        image = face_recognition.load_image_file(img_path)
        boxes = face_recognition.face_locations(image)

        # Skip images without exactly one face

        if len(boxes) != 1:
            continue

        encoding = face_recognition.face_encodings(image)[0]
        encodings.append(encoding)
        names.append(person_name)

Training the SVM Classifier

Pass the list of encodings and labels to svm.SVC from scikit-learn. The gamma='scale' parameter automatically determines the kernel coefficient, and setting probability=True enables confidence scores if needed later.

# Train the SVM classifier

clf = svm.SVC(gamma="scale", probability=True)
clf.fit(encodings, names)

The classifier uses a one-vs-one strategy to handle multiple classes, creating decision boundaries that separate each person's embedding cluster from every other cluster.

Running Multi-Face Recognition Inference

To identify multiple people in a new image, detect all faces, compute their encodings, and call clf.predict() for each face. This workflow is demonstrated in the official example script examples/face_recognition_svm.py.

# Recognize faces in a new image

test_image = face_recognition.load_image_file("test_image.jpg")
face_locations = face_recognition.face_locations(test_image)
face_encodings = face_recognition.face_encodings(test_image, face_locations)

for location, encoding in zip(face_locations, face_encodings):
    name = clf.predict([encoding])[0]
    top, right, bottom, left = location
    print(f"Found {name} at ({left}, {top}) - ({right}, {bottom})")

Summary

• The face_recognition library generates 128-dimensional embeddings using dlib's deep learning model, implemented in face_recognition/api.py.
• SVM classifiers from scikit-learn work effectively on these embeddings for multi-face recognition, especially with small training datasets.
• Organize training data in labeled subdirectories, processing only images with exactly one face to avoid ambiguous training labels.
• The official examples/face_recognition_svm.py provides a complete reference implementation combining face encoding and SVM training.

Frequently Asked Questions

How many images per person are needed to train the SVM?

You can achieve reasonable accuracy with as few as 5-10 images per person, though 20-30 images yield more robust results. The SVM performs well on small datasets because the 128-dimensional embeddings from face_recognition are already highly discriminative.

Can I use this approach for real-time video recognition?

Yes, but you should optimize the face detection step. Use face_locations with the model="hog" parameter for faster CPU processing, or model="cnn" for higher accuracy on GPU. The SVM prediction itself is extremely fast (milliseconds), so the bottleneck is typically face detection and encoding.

What if the SVM predicts the wrong person?

Check your training data quality first. Ensure training images contain only the target person with clear, front-facing views. If problems persist, you can add a distance threshold by calculating the Euclidean distance between the test encoding and the closest training encoding of the predicted class. If the distance exceeds a threshold (typically 0.6), classify the face as "unknown."

Is there an alternative to SVM for this task?

You can replace the SVM with other classifiers like k-Nearest Neighbors (k-NN), Random Forest, or Logistic Regression. k-NN is particularly popular for face recognition because it simply stores all training embeddings and uses distance metrics at inference time, requiring no training phase. However, SVMs generally offer better generalization when you have limited training examples per person.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}