How to Implement Face Detection and Recognition with ailia-models: A Complete Guide

To implement face detection and recognition with ailia-models, use the detection scripts in face_detection/ (RetinaFace, BlazeFace, YOLOv3-face) to locate and crop faces, then pass those crops to identification models in face_identification/ (ArcFace, FaceNet, CosFace) to extract embeddings and compute cosine similarity for identity matching.

The axinc-ai/ailia-models repository provides production-ready deep learning models wrapped in a lightweight Python SDK (ailia). For face detection and recognition with ailia-models, the repository offers specialized pipelines that handle everything from bounding box detection to facial embedding extraction and identity tracking.

Face Detection Pipeline

The detection layer locates faces in images or video streams and returns bounding boxes with optional landmarks. The repository implements several architectures, all following a consistent preprocessing and post-processing pattern.

RetinaFace Implementation

The RetinaFace detector (face_detection/retinaface/retinaface.py) demonstrates the standard detection flow:

1. Model Initialization – The script uses arg_utils.get_base_parser for CLI arguments and model_utils.check_and_download_models to fetch retinaface_resnet50.onnx (or the mobile variant) from remote storage.

2. Preprocessing – Input images undergo mean subtraction with values (104, 117, 123) to convert to BGR-normalized format expected by the ONNX model. The --rescale argument optionally scales down large images to fit memory constraints.

3. Inference – An ailia.Net instance loads the model, and net.predict([input_data]) runs the forward pass.

4. Post-processing – Raw predictions (loc, conf, landms) pass through retinaface_utils.decode and decode_landm. The postprocessing() function scales boxes back to original image coordinates, applies confidence filtering (CONFIDENCE_THRES), sorts detections, and runs py_cpu_nms to eliminate overlapping boxes.

Alternative Detectors

Depending on your speed and accuracy requirements, you can substitute RetinaFace with:

• BlazeFace (face_detection/blazeface/blazeface.py) – Lightweight models with front and back network variants (use --back for 256×256 resolution).
• YOLOv3-face (face_detection/yolov3-face/yolov3-face.py) – Classic YOLO-style detection optimized for facial features.
• DBFace (face_detection/dbface/dbface.py) – High-accuracy detection with custom decoding utilities.

Face Recognition and Identification Pipeline

The identification layer converts cropped face regions into compact embeddings and compares them to determine identity. This process typically follows detection in a cascaded workflow.

ArcFace Recognition Flow

The ArcFace implementation (face_identification/arcface/arcface.py) provides the most complete reference for face recognition with ailia-models:

1. Cascade Detection – ArcFace first runs a detector specified via the --face argument (defaulting to YOLOv3-face) to obtain cropped faces. The get_faces() function returns a dictionary containing the resized frame and original coordinates for each detection.

2. Embedding Extraction – The preprocess_image function applies histogram equalization, horizontal flipping, and normalization to [-1, 1]. The script loads arcface.onnx via ailia.Net and feeds the preprocessed tensor. For batch processing, it may concatenate tensor copies to meet the model's required batch dimensions.

3. Similarity Scoring – The cosin_metric() function computes cosine similarity between embeddings. The face_identification() logic compares new detections against existing tracks, using the highest historical similarity. If the score falls below the configurable --threshold (default 0.2557), the system assigns a new identity; otherwise, it inherits the existing ID.

4. Visualization – The display_tracks() function draws colored bounding boxes (HSV-based colors per ID) and maintains a side-panel showing recent crops for each tracked identity.

Other Recognition Models

The repository provides alternatives to ArcFace:

• FaceNet (face_identification/facenet_pytorch/facenet_pytorch.py) – PyTorch-converted models using different embedding strategies.
• CosFace and InsightFace – Alternative architectures following the same detect-then-recognize pattern with model-specific preprocessing.

Integrating Detection and Recognition

A complete face detection and recognition pipeline with ailia-models follows this workflow:

1. Run any detector to obtain bounding boxes and crop face regions.
2. Pass crops through a recognition model to generate fixed-length embeddings.
3. Compare embeddings using cosine similarity or Euclidean distance.
4. Assign or update identity IDs based on threshold comparisons.
5. Visualize results or export JSON via --write_json.

The compare_images() and compare_video() functions in arcface.py implement this cascade for both static images and video streams, handling the entire lifecycle from detection to persistent tracking.

Practical Code Examples

Command Line Face Detection

Detect faces in a static image using RetinaFace with ResNet50 backbone:

python face_detection/retinaface/retinaface.py \
    --input selfie.jpg \
    --savepath result.png \
    --arch resnet50

Use --arch mobile0.25 for faster inference on resource-constrained devices.

Real-Time Video Detection

Process webcam input with BlazeFace using the higher-resolution back network:

python face_detection/blazeface/blazeface.py \
    --video 0 \
    --savepath out.mp4 \
    --back

Omit --back to use the lightweight front-net optimized for mobile deployment.

Face Identification Comparison

Compare two faces using ArcFace to determine if they belong to the same person:

python face_identification/arcface/arcface.py \
    --inputs person_a.jpg person_b.jpg \
    --arch arcface

The script outputs the cosine similarity score and match determination based on the default threshold.

Webcam-Based Recognition with Tracking

Run real-time identification with persistent ID tracking:

python face_identification/arcface/arcface.py \
    --video 0 \
    --face yolov3

Press q to exit the live visualization window showing colored bounding boxes and identity labels.

Programmatic SDK Usage

Integrate the pipeline directly in Python without CLI invocation:

import cv2
import ailia
from face_detection.retinaface import retinaface_utils as rut
from face_identification.arcface.arcface import preprocess_image, cosin_metric

# Initialize detector

detector = ailia.Net(
    'face_detection/retinaface/retinaface_resnet50.onnx.prototxt',
    'face_detection/retinaface/retinaface_resnet50.onnx',
    env_id=0
)

# Initialize recognizer

recognizer = ailia.Net(
    'face_identification/arcface/arcface.onnx.prototxt',
    'face_identification/arcface/arcface.onnx',
    env_id=0
)

# Load and preprocess image

frame = cv2.imread('group.jpg')
img = frame - (104, 117, 123)  # Mean subtraction

img = img.transpose(2, 0, 1).astype('float32')

# Detect faces

detector.compute(img)
loc = detector.get_output(0)
conf = detector.get_output(1)
landms = detector.get_output(2)

# Decode and extract embeddings

embeddings = []
for box in rut.decode(loc, conf, landms):  # Simplified for brevity

    x1, y1, x2, y2 = map(int, box[:4])
    crop = frame[y1:y2, x1:x2]
    inp = preprocess_image(crop)
    emb = recognizer.predict([inp])[0]
    embeddings.append(emb)

# Compute pairwise similarities

for i in range(len(embeddings)):
    for j in range(i+1, len(embeddings)):
        sim = cosin_metric(embeddings[i], embeddings[j])
        print(f'Similarity {i}-{j}: {sim:.3f}')

Key Source Files and Architecture

Component	File Path	Purpose
RetinaFace Detector	face_detection/retinaface/retinaface.py	End-to-end detection with ResNet50/MobileNet backbones
BlazeFace Detector	face_detection/blazeface/blazeface.py	Lightweight detection with front/back network options
YOLOv3-face	face_detection/yolov3-face/yolov3-face.py	YOLO-based detection for cascade usage
ArcFace Recognizer	face_identification/arcface/arcface.py	Complete pipeline: detection → embedding → tracking
FaceNet	face_identification/facenet_pytorch/facenet_pytorch.py	PyTorch-based embedding extraction
Utility Modules	utils/detector_utils.py, utils/model_utils.py	Model downloading, NMS, and decoding helpers
Preprocessing	face_identification/arcface/arcface.py (preprocess_image)	Histogram equalization and tensor formatting

Summary

• ailia-models separates face detection (face_detection/) from recognition (face_identification/), allowing flexible pipeline construction.
• RetinaFace and BlazeFace provide robust detection with NMS post-processing via py_cpu_nms.
• ArcFace implements the full cascade: it accepts a --face detector argument, extracts embeddings using preprocess_image, and tracks identities using cosin_metric() with a default threshold of 0.2557.
• All models use the ailia.Net class for inference, supporting both CPU and GPU backends via the env_id parameter.
• The repository includes complete CLI tools and utility functions (arg_utils, model_utils) for rapid deployment without custom code.

Frequently Asked Questions

What is the difference between face detection and face recognition in ailia-models?

Face detection locates faces within an image and returns bounding boxes (handled by modules in face_detection/ such as RetinaFace or BlazeFace). Face recognition converts cropped face images into numerical embeddings and compares them to determine identity (handled by face_identification/ models like ArcFace). Detection finds where faces are; recognition determines who they belong to.

Which model should I use for real-time face detection?

For real-time applications, use BlazeFace (face_detection/blazeface/blazeface.py) with the front-net (omit --back) or RetinaFace with --arch mobile0.25. Both offer optimized inference speeds suitable for webcam processing while maintaining sufficient accuracy for most tracking scenarios.

How do I adjust the similarity threshold for face recognition?

Pass the --threshold argument when running ArcFace (default is 0.2557). Lower values make the system more strict (fewer false matches), while higher values allow more lenient matching. The cosin_metric() function in arcface.py computes similarity scores between 0 and 1, where values above your threshold indicate a match.

Can I use different detectors with the ArcFace recognizer?

Yes. ArcFace accepts a --face argument to specify the detector backend. Valid options include yolov3 (default), blazeface, or retinaface. The get_faces() function in arcface.py automatically handles the different output formats from each detector to provide standardized crops for recognition.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}