# How to Implement Real-Time Face Recognition from Video Streams with Python

> Learn to implement real-time face recognition from video streams using Python, OpenCV and the face_recognition library. Detect, encode, and match faces efficiently.

- Repository: [Adam Geitgey/face_recognition](https://github.com/ageitgey/face_recognition)
- Tags: how-to-guide
- Published: 2026-03-06

---

**Real-time face recognition from video streams is achieved by capturing frames with OpenCV, converting BGR to RGB, detecting faces with `face_recognition.face_locations`, encoding them with `face_recognition.face_encodings`, and matching against known encodings using `face_recognition.compare_faces` or `face_recognition.face_distance`.**

The `face_recognition` library provides a high-level Python API built on top of dlib's state-of-the-art face recognition models, making it straightforward to process live video feeds. By combining OpenCV for video I/O with the library's optimized face detection and encoding pipeline, you can build systems that identify individuals in real-time with minimal latency.

## The Real-Time Face Recognition Pipeline

Real-time processing requires efficient coordination between video capture, preprocessing, detection, and matching. The `face_recognition` library exposes specific functions in [`face_recognition/api.py`](https://github.com/ageitgey/face_recognition/blob/main/face_recognition/api.py) that handle each stage.

### Video Capture and Frame Preprocessing

OpenCV's `cv2.VideoCapture` interface pulls frames from webcams or video files. Since OpenCV uses BGR color ordering while `face_recognition` expects RGB, you must convert each frame using `frame[:, :, ::-1]` or `cv2.cvtColor`.

For performance-critical applications, resize frames to ¼ size before analysis. This reduces the pixel count by 16× while preserving enough detail for detection. The original frame is retained for display purposes, with detection coordinates scaled back up for annotation.

### Face Detection

The `face_locations` function in [`face_recognition/api.py`](https://github.com/ageitgey/face_recognition/blob/main/face_recognition/api.py) wraps `_raw_face_locations` and returns bounding boxes as `(top, right, bottom, left)` tuples. It supports two models:

- **HOG (Histogram of Oriented Gradients)**: Default CPU-based detector. Fast and sufficient for most real-time applications.
- **CNN (Convolutional Neural Network)**: More accurate but requires GPU acceleration via CUDA to achieve real-time speeds.

### Face Encoding and Matching

Once faces are located, `face_encodings` computes 128-dimensional embeddings using the pretrained model at `face_recognition_models.face_recognition_model_location`. These embeddings are numerical fingerprints that remain consistent across lighting and angle variations.

Matching occurs via:

- **`compare_faces`**: Returns a boolean list indicating which known encoding matches the target (using a default tolerance of 0.6).
- **`face_distance`**: Returns Euclidean distances for fine-grained confidence scoring. Use `numpy.argmin` to select the best match when multiple candidates exist.

## Complete Implementation Examples

### Basic Webcam Implementation

This straightforward approach processes every frame at full resolution. Suitable for proof-of-concept or when hardware resources are abundant.

```python
import face_recognition
import cv2
import numpy as np

video_capture = cv2.VideoCapture(0)

# Load and encode known faces

obama_image = face_recognition.load_image_file("obama.jpg")
obama_encoding = face_recognition.face_encodings(obama_image)[0]

biden_image = face_recognition.load_image_file("biden.jpg")
biden_encoding = face_recognition.face_encodings(biden_image)[0]

known_encodings = [obama_encoding, biden_encoding]
known_names = ["Barack Obama", "Joe Biden"]

while True:
    ret, frame = video_capture.read()
    rgb_frame = frame[:, :, ::-1]  # Convert BGR to RGB

    face_locations = face_recognition.face_locations(rgb_frame)
    face_encodings = face_recognition.face_encodings(rgb_frame, face_locations)

    for (top, right, bottom, left), face_encoding in zip(face_locations, face_encodings):
        matches = face_recognition.compare_faces(known_encodings, face_encoding)
        name = "Unknown"

        if True in matches:
            first_match_index = matches.index(True)
            name = known_names[first_match_index]

        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
        cv2.putText(frame, name, (left + 6, bottom - 6),
                    cv2.FONT_HERSHEY_DUPLEX, 1.0, (255, 255, 255), 1)

    cv2.imshow('Video', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

video_capture.release()
cv2.destroyAllWindows()

```

*Source: [examples/facerec_from_webcam.py](https://github.com/ageitgey/face_recognition/blob/master/examples/facerec_from_webcam.py)*

### Optimized Real-Time Implementation

For production real-time face recognition from video streams, process every other frame and analyze a quarter-resolution copy. This approach, demonstrated in [`examples/facerec_from_webcam_faster.py`](https://github.com/ageitgey/face_recognition/blob/main/examples/facerec_from_webcam_faster.py), sustains approximately 15 FPS on standard laptop CPUs.

```python
import face_recognition
import cv2
import numpy as np

video_capture = cv2.VideoCapture(0)

# Initialize known faces

obama_image = face_recognition.load_image_file("obama.jpg")
obama_encoding = face_recognition.face_encodings(obama_image)[0]

biden_image = face_recognition.load_image_file("biden.jpg")
biden_encoding = face_recognition.face_encodings(biden_image)[0]

known_encodings = [obama_encoding, biden_encoding]
known_names = ["Barack Obama", "Joe Biden"]

face_locations = []
face_names = []
process_this_frame = True

while True:
    ret, frame = video_capture.read()
    
    if process_this_frame:
        # Resize to 1/4 size for faster processing

        small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
        rgb_small_frame = small_frame[:, :, ::-1]  # BGR to RGB

        
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)
        
        face_names = []
        for face_encoding in face_encodings:
            matches = face_recognition.compare_faces(known_encodings, face_encoding)
            name = "Unknown"
            
            # Use distance to find best match

            face_distances = face_recognition.face_distance(known_encodings, face_encoding)
            best_match_index = np.argmin(face_distances)
            if matches[best_match_index]:
                name = known_names[best_match_index]
            
            face_names.append(name)
    
    process_this_frame = not process_this_frame  # Skip every other frame

    
    # Display results (scale back up to original size)

    for (top, right, bottom, left), name in zip(face_locations, face_names):
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4
        
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        cv2.putText(frame, name, (left + 6, bottom - 6),
                    cv2.FONT_HERSHEY_DUPLEX, 1.0, (255, 255, 255), 1)
    
    cv2.imshow('Video', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

video_capture.release()
cv2.destroyAllWindows()

```

*Source: [examples/facerec_from_webcam_faster.py](https://github.com/ageitgey/face_recognition/blob/master/examples/facerec_from_webcam_faster.py)*

## Key Source Files and API Reference

Understanding the underlying implementation helps optimize your real-time face recognition system:

| File | Purpose | Key Functions |
|------|---------|---------------|
| [`face_recognition/api.py`](https://github.com/ageitgey/face_recognition/blob/main/face_recognition/api.py) | Core library API | `face_locations()`, `face_encodings()`, `compare_faces()`, `face_distance()` |
| [`examples/facerec_from_webcam.py`](https://github.com/ageitgey/face_recognition/blob/main/examples/facerec_from_webcam.py) | Basic implementation | Full-resolution processing example |
| [`examples/facerec_from_webcam_faster.py`](https://github.com/ageitgey/face_recognition/blob/main/examples/facerec_from_webcam_faster.py) | Optimized implementation | Quarter-resolution + frame skipping |
| [`face_recognition/face_recognition_cli.py`](https://github.com/ageitgey/face_recognition/blob/main/face_recognition/face_recognition_cli.py) | Command-line interface | Batch processing utilities |

In [`face_recognition/api.py`](https://github.com/ageitgey/face_recognition/blob/main/face_recognition/api.py), the `face_locations` function wraps `_raw_face_locations` and normalizes outputs to CSS-style `(top, right, bottom, left)` tuples. The `face_encodings` function generates 128-dimensional embeddings using the pretrained model located via `face_recognition_models.face_recognition_model_location`.

## Summary

- **Real-time face recognition from video streams** requires converting OpenCV's BGR frames to RGB, detecting faces with `face_locations`, and matching embeddings via `compare_faces` or `face_distance`.
- **Performance optimization** involves resizing frames to ¼ resolution before processing and analyzing only every other frame, as demonstrated in [`examples/facerec_from_webcam_faster.py`](https://github.com/ageitgey/face_recognition/blob/main/examples/facerec_from_webcam_faster.py).
- **Detection models** include the default **HOG** (CPU-optimized) and optional **CNN** (GPU-accelerated for higher accuracy).
- **Core API functions** reside in [`face_recognition/api.py`](https://github.com/ageitgey/face_recognition/blob/main/face_recognition/api.py), providing `face_encodings` for 128-dimensional embeddings and `face_distance` for confidence scoring.

## Frequently Asked Questions

### What is the difference between HOG and CNN face detection models?

The **HOG** (Histogram of Oriented Gradients) model is the default CPU-based detector that prioritizes speed over absolute accuracy, making it ideal for real-time applications on standard hardware. The **CNN** (Convolutional Neural Network) model offers superior accuracy and handles challenging angles better, but requires GPU acceleration via CUDA to achieve real-time frame rates. You can specify the model in `face_locations` via the `model` parameter (e.g., `model="cnn"`).

### How can I improve the frame rate of my real-time face recognition system?

To achieve higher FPS, implement the optimizations found in [`examples/facerec_from_webcam_faster.py`](https://github.com/ageitgey/face_recognition/blob/main/examples/facerec_from_webcam_faster.py): **resize frames to ¼ size** (reducing pixel count by 16×) before passing them to `face_locations` and `face_encodings`, and **process only every other frame** while displaying annotations on the original full-resolution stream. These techniques can sustain approximately 15 FPS on standard laptop CPUs without sacrificing detection accuracy.

### What does the 128-dimensional face encoding represent?

The 128-dimensional embedding generated by `face_recognition.face_encodings` is a numerical fingerprint that uniquely represents the facial features of a detected face. This vector is computed by a pretrained dlib model (located via `face_recognition_models.face_recognition_model_location`) and is designed to be consistent across variations in lighting, pose, and expression. Faces belonging to the same person will produce embeddings with small Euclidean distances, while different individuals will have larger distances.

### How do I handle unknown faces in the video stream?

When `face_recognition.compare_faces` returns `False` for all known encodings, or when `face_recognition.face_distance` exceeds your tolerance threshold (default 0.6), classify the detection as "Unknown". For robust matching, use `face_distance` combined with `numpy.argmin` to find the closest match only if the distance is below your threshold; otherwise label as unknown. This prevents false positives when the video stream contains people not in your known database.