1.091.2 Face Detection & Recognition Libraries#
MediaPipe, dlib, InsightFace, OpenCV YuNet/SFace and the hosted APIs, compared on capability, on what the weights are licensed to do, and on biometric law.
At a glance#
| Library | Verdict | Latest release |
|---|---|---|
| MediaPipe (Google) | 1.0.1 (2026-08-14). Detection plus 478 3D landmarks, 52 blendshapes and a pose matrix, with first-party Android, iOS and Web bindings — the only option that does this job. No recognition model at all, and the license chain came back clean end to end. Google’s own benchmark: BlazeFace short-range at 2.94 ms CPU on a Pixel 6. | 1.0.1 · 2026-08-14 |
| OpenCV — YuNet and SFace | The correction this refresh exists for. cv2.FaceDetectorYN runs YuNet (MIT weights, 227 KB, 0.7503 WIDER FACE hard val) and cv2.FaceRecognizerSF runs SFace (Apache-2.0 weights, 37 MB, 0.9940). The only detection-plus-recognition pair whose weights carry license files permitting commercial use — and it is inside the OpenCV most readers already import. | 5.0.0.93 · 2026-07-29 |
| dlib | 20.0.1. One library, three license positions: the HOG detector’s weights are Boost-licensed inside the header; the recognition ResNet is public domain per its author at 0.993833 LFW; and the 68-point landmark model carries a note from that same author saying it ‘can’t be used in a commercial product’ because of the iBUG 300-W dataset. PyPI ships one 3.3 MB sdist and no wheels — every install compiles C++. | 20.0.1 · 2026-03-29 |
| InsightFace | 1.0.1 (2026-05-23), after three years without a release. MIT code with no commercial limitation; ALL models — including the buffalo_l pack the library auto-downloads — are ‘for non-commercial research purposes only’ by the project’s own statement in three separate READMEs. Licensing is by email since 2025-11-24. Highest accuracy on the benchmarks that separate models, and the only project publishing its own demographic breakdown. | 1.0.1 · 2026-05-23 |
| RetinaFace | A paper (91.4 AP on WIDER FACE hard TEST, by its own authors) and a chain of MIT re-implementations over weights that entered from InsightFace. The MobileNet variant people deploy scores 80.99 hard val by its own repository’s description. RetinaFace is an InsightFace project, not a competitor to it. | 1.1.1 · 2022-07-11 |
| MTCNN | 1.0.0 (2024-10-08). Zero open pull requests and zero merged — finished rather than stalled. ~2 MB, comprehensively superseded on accuracy, and still reachable through facenet-pytorch and DeepFace. Nothing will break; nothing will be fixed. | 1.0.0 · 2024-10-08 |
| DeepFace and face_recognition | The wrapper tier S1 omits. DeepFace (23,337 stars, last commit 2026-08-24) is maintained and its README states the trap outright: ‘License types will be inherited when you intend to utilize those models.’ face_recognition has 56,682 stars — the most in the category — and no release since 2020-02-20, with 57 open pull requests and one merged in six months. | 0.0.100 · 2026-05-09 |
| Amazon Rekognition, Face++ and Azure Face | The hosted tier, where you choose a counterparty rather than a model. Rekognition is $1.00 per 1,000 images in the first tier. Azure sells detection to anyone and gates identification and verification to ‘customers managed by Microsoft’. Megvii, behind Face++, was added to the US Entity List effective 2019-10-09. | — |
Latest release observed from PyPI in 2026-09.
What the research found
- The license attaches at three separate links — package, weights, training data — and in this category they disagree: the top recognition model is research-only, the most reproduced landmark model carries its author’s note against commercial use, and a dependency scanner reads only the first link
- OpenCV has shipped YuNet (MIT, 227 KB) and SFace (Apache-2.0, 37 MB) in the core library since 4.5.4 — the only permissively-licensed detection-plus-recognition pair in the category, and the thing an earlier pass of this survey missed entirely
- Detection, landmarks and recognition are three jobs with three benchmark families; a WIDER FACE AP and an LFW accuracy cannot share a column, and LFW no longer separates the recognition models at all (99.83 / 99.40 / 99.38)
- The encoder is the irreversible choice: detectors swap in an afternoon, but once templates are stored, changing the encoder means every enrolled person enrolls again — keep it behind an interface from the first commit
- Every accuracy figure in this category is self-reported by the project that published the model; no open-source model here participates in NIST FRTE, and the one project that publishes a demographic breakdown shows a ~20-point spread across subsets on its flagship model
Explainer
Face Detection: Performance & User Experience Fundamentals#
Purpose: Bridge general technical knowledge to face detection library decision-making Audience: Developers/engineers familiar with basic computer vision concepts Context: Why face detection library choice directly impacts user experience and system performance
Beyond Basic Understanding#
The User Experience Reality#
Finding faces in images is where face detection starts. Where it bites is direct application responsiveness:
# Real-time video application performance
target_fps = 30
frame_time_budget = 33.3 # milliseconds per frame (1000ms / 30fps)
# Performance scenarios:
haar_cascade_latency = 8 # 8ms per frame (125 FPS capable)
dlib_hog_latency = 15 # 15ms per frame (66 FPS capable)
retinaface_latency = 85 # 85ms per frame (11 FPS - DROPS FRAMES)
# User experience impact:
# 30 FPS: Smooth, professional experience
# 15 FPS: Choppy, noticeable lag
# 11 FPS: Unusable for real-time interaction
# Business impact calculation:
video_conference_users = 10_000
user_churn_rate_bad_performance = 0.35 # 35% abandon app due to lag
monthly_subscription = 15
monthly_revenue_loss = video_conference_users * user_churn_rate_bad_performance * monthly_subscription
# = $52,500 lost monthly recurring revenue from wrong algorithm choiceWhen Face Detection Becomes Critical#
Modern applications hit face detection bottlenecks in predictable patterns:
- Security systems: 24/7 real-time monitoring, missed detection = security breach
- Photo organization: Batch processing thousands of images, accuracy determines usability
- AR filters/effects: Real-time 3D mesh required, latency = broken immersion
- Attendance systems: Single frame accuracy critical, false negative = missed attendance
- Authentication: Security vs convenience trade-off, false positive = breach risk
Business Impact Calculations#
False Negative Impact (Security Camera):
# Security monitoring system
cameras = 50
hours_per_day = 24
people_per_hour = 12
# Detection accuracy scenarios:
high_accuracy_detection = 0.98 # RetinaFace
fast_detection = 0.92 # Haar Cascade
# Missed detections per day:
high_accuracy_misses = cameras * hours_per_day * people_per_hour * (1 - high_accuracy_detection)
# = 288 missed detections per day
fast_detection_misses = cameras * hours_per_day * people_per_hour * (1 - fast_detection)
# = 1,152 missed detections per day
# Security incident risk:
# 4x more missed detections = significantly higher security risk
# Cost of single security incident: $50,000 - $500,000+Latency Impact (Photo Booth Application):
# Event photo booth usage
photos_per_event = 200
events_per_month = 30
detection_time_fast = 0.010 # 10ms - Haar
detection_time_accurate = 0.100 # 100ms - RetinaFace
# User wait time:
monthly_user_wait_fast = photos_per_event * events_per_month * detection_time_fast
# = 60 seconds total monthly wait time
monthly_user_wait_accurate = photos_per_event * events_per_month * detection_time_accurate
# = 600 seconds (10 minutes) monthly wait time
# User satisfaction impact:
# Sub-second response: 95% satisfaction
# Multi-second response: 67% satisfaction
# Wrong algorithm = 28% satisfaction drop = poor reviewsModel Size Impact (Mobile Application):
# Mobile app deployment
app_size_without_detection = 25 # MB base app
model_sizes = {
'haar_cascade': 0.9, # 900 KB
'dlib_cnn': 10.5, # 10.5 MB
'mediapipe': 3.2, # 3.2 MB
'retinaface': 26.8 # 26.8 MB
}
# Download conversion rates:
app_size_threshold = 50 # MB before users abandon
# Haar/MediaPipe: Under threshold, high conversion
# RetinaFace: 51.8 MB total - exceeds threshold, conversion drops 40%
# Monthly install impact:
monthly_installs_potential = 50_000
conversion_rate_small = 0.85
conversion_rate_large = 0.51 # 40% drop for large apps
lost_installs = monthly_installs_potential * (conversion_rate_small - conversion_rate_large)
# = 17,000 lost installs per month from wrong model size choiceCore Face Detection Algorithm Categories#
1. Traditional Methods (Haar Cascades, HOG)#
What they prioritize: Speed and computational efficiency Trade-off: Lower accuracy for real-time performance Real-world uses: Security cameras, embedded systems, legacy hardware
Performance characteristics:
# Haar Cascade example - why speed matters
camera_feed_fps = 30
frame_resolution = (640, 480)
# Haar Cascade detection:
detection_time = 5_ms # Extremely fast
detections_per_second = 200 # Can process 6x real-time
cpu_usage = 15 # Percent, single core
# Accuracy trade-offs:
frontal_face_accuracy = 0.95 # Excellent for direct faces
profile_face_accuracy = 0.45 # Poor for side views
occluded_face_accuracy = 0.60 # Struggles with partial faces
# Use case: 24/7 security monitoring
power_consumption = 5_watts # Low power, can run continuously
annual_electricity_cost = 5 * 24 * 365 / 1000 * 0.12 # $5.26 per camera
# Scales to hundreds of cameras economicallyThe Speed Priority:
- Real-time video: Essential for webcam applications (30+ FPS)
- Embedded systems: Raspberry Pi, mobile devices with limited compute
- Cost efficiency: Minimal CPU/GPU requirements = lower cloud costs
- Battery life: Mobile applications need power-efficient detection
HOG (Histogram of Oriented Gradients):
# Dlib HOG detector characteristics
detection_time = 15_ms # Still real-time capable
accuracy_improvement = 1.15 # 15% better than Haar
memory_usage = 8_MB # Lightweight model
# When to choose HOG over Haar:
# - Need better accuracy but still real-time
# - Faces at various angles (not just frontal)
# - Acceptable to use slightly more CPU
# - Desktop applications (not embedded)
# Real-world comparison:
haar_false_positives = 12 # Per 100 detections
hog_false_positives = 6 # 50% reduction
# Fewer false alarms in security systems = better UX2. Deep Learning Methods (CNN, Cascade Networks)#
What they prioritize: Detection accuracy over speed Trade-off: Higher computational cost for better accuracy Real-world uses: Photo processing, high-accuracy requirements, cloud services
MTCNN (Multi-task Cascaded Convolutional Networks):
# Three-stage cascade approach
stage1_proposals = 1000 # Rapid elimination of non-faces
stage2_refinement = 100 # Refine promising regions
stage3_final = 5 # Precise bounding boxes + landmarks
# Performance profile:
detection_time = 45_ms # Too slow for real-time (22 FPS)
accuracy = 0.95 # Excellent accuracy
false_positive_rate = 0.02 # Very low false alarms
# Cascade efficiency:
# Stage 1: 5ms, eliminates 90% of image regions
# Stage 2: 20ms, processes only 10% of regions
# Stage 3: 20ms, final refinement on <1% of regions
# Result: 20x faster than processing entire image with accurate model
# Use case: Batch photo processing
photos_per_batch = 1000
processing_time = 1000 * 0.045 # 45 seconds total
# Acceptable for overnight processing, unacceptable for real-timeRetinaFace (State-of-the-art CNN):
# Highest accuracy commercial option
detection_accuracy = 0.98 # Industry-leading accuracy
detection_time_cpu = 850_ms # Unusable for real-time on CPU
detection_time_gpu = 65_ms # Borderline real-time on GPU
# WIDER FACE benchmark (industry standard):
easy_subset = 0.99 # 99% accuracy on clear faces
medium_subset = 0.97 # 97% on partially occluded
hard_subset = 0.91 # 91% on difficult conditions
# When RetinaFace is worth the cost:
# - Cloud-based batch processing with GPUs
# - Critical accuracy applications (law enforcement)
# - Photo album organization (offline processing)
# - When you can't afford missed detections
# Cost analysis:
gpu_instance_cost = 0.50 # Per hour (AWS g4dn.xlarge)
images_processed_per_hour = 5000 # With GPU acceleration
cost_per_image = 0.0001 # $0.0001 per image
# Compare to fast model:
cpu_instance_cost = 0.05 # Per hour (t3.medium)
fast_model_images_per_hour = 8000 # Haar on CPU
fast_cost_per_image = 0.00000625 # 16x cheaper but less accurate3. Modern Approaches (MediaPipe, Mobile-Optimized Networks)#
What they prioritize: Balance of speed, accuracy, and deployment efficiency Trade-off: Optimized for specific platforms (mobile, web) Real-world uses: AR applications, mobile apps, browser-based detection
Google MediaPipe:
# Mobile-optimized face detection + mesh
detection_time_mobile = 12_ms # Excellent mobile performance
landmark_points = 468 # Full 3D face mesh
model_size = 3.2_MB # Small enough for mobile apps
# Battery efficiency:
traditional_cnn_power = 2500_mW # Drains battery quickly
mediapipe_power = 450_mW # 5.5x more efficient
# Users can run AR filters for hours vs minutes
# 3D mesh capabilities:
# - Real-time AR effects (Snapchat-style filters)
# - Head pose estimation (gaze tracking)
# - Facial animation (avatar control)
# - Depth-aware effects (lighting, occlusion)
# Use case: Social media AR filters
users_per_day = 100_000
average_session_time = 3 # minutes
total_compute_time = 300_000 # minutes
# Must run on user devices (not cloud) = need efficient model
# MediaPipe: Only option for large-scale mobile ARBlazeFace (MediaPipe component):
# Specialized for mobile front-camera detection
detection_time = 6_ms # Fastest mobile option
accuracy_frontal = 0.94 # Optimized for selfies
accuracy_profile = 0.72 # Lower for side views
# Design trade-offs:
# Assumes: Front-facing camera, good lighting, close-up faces
# Result: 2-3x faster than general-purpose detectors
# Perfect for: Selfie apps, video calls, AR filters
# Wrong for: Security cameras, group photos, varied angles
# Mobile deployment advantages:
model_quantization = 'int8' # 4x smaller, minimal accuracy loss
on_device_inference = True # Privacy, no server costs
offline_capability = True # Works without internet4. Cloud-based APIs (Face++, Amazon Rekognition, Azure Face)#
What they prioritize: Zero infrastructure management, high accuracy Trade-off: Latency, privacy, ongoing costs Real-world uses: MVPs, low-volume applications, full-service solutions
# Cloud API economics
api_cost_per_call = 0.001 # $1 per 1000 detections
monthly_detections = 500_000
monthly_api_cost = 500 # $500/month
# Self-hosted GPU alternative:
gpu_instance_monthly = 360 # $360/month (24/7 g4dn.xlarge)
# Break-even point: 360,000 detections/month
# Decision framework:
if monthly_detections < 360_000:
use_cloud_api() # More cost-effective
elif monthly_detections > 360_000:
self_host_gpu() # Better economics at scale
# Hidden costs of cloud APIs:
network_latency = 50_ms # Best case latency
privacy_compliance = 'complex' # Sending user photos to 3rd party
vendor_lock_in_risk = 'high' # Hard to migrate
rate_limiting = True # Throttling at high volume
# When cloud APIs make sense:
# - MVP/prototype stage
# - <100k detections/month
# - No real-time requirements
# - No privacy restrictions
# - Want additional features (age, emotion detection)Performance Characteristics#
Detection Accuracy Benchmarks#
WIDER FACE Dataset (Industry Standard):
# Three difficulty categories simulate real-world conditions
wider_face_subsets = {
'easy': {
'characteristics': 'Large faces, frontal view, minimal occlusion',
'face_size': '>100px',
'example_scenarios': 'Portrait photos, ID photos, close-up selfies'
},
'medium': {
'characteristics': 'Medium faces, some occlusion, varied angles',
'face_size': '50-100px',
'example_scenarios': 'Group photos, casual photos, security footage'
},
'hard': {
'characteristics': 'Small faces, heavy occlusion, extreme angles',
'face_size': '<50px',
'example_scenarios': 'Crowd surveillance, distant cameras, poor conditions'
}
}
# Algorithm performance on WIDER FACE:
benchmark_results = {
'Haar Cascade': {'easy': 0.85, 'medium': 0.60, 'hard': 0.30},
'Dlib HOG': {'easy': 0.89, 'medium': 0.68, 'hard': 0.38},
'MTCNN': {'easy': 0.95, 'medium': 0.88, 'hard': 0.72},
'RetinaFace': {'easy': 0.99, 'medium': 0.97, 'hard': 0.91},
'MediaPipe': {'easy': 0.96, 'medium': 0.89, 'hard': 0.73}
}
# Why "hard" subset matters for production:
# Real-world conditions are rarely "easy"
# - Security cameras: Often distant, angled, partially occluded
# - Photo albums: Mix of all conditions
# - Surveillance: Worst-case scenarios are most important
# If hard subset < 0.7, expect production accuracy issuesPrecision vs Recall Trade-off:
# Understanding detection trade-offs
precision = true_positives / (true_positives + false_positives)
recall = true_positives / (true_positives + false_negatives)
# Security application example:
security_high_recall = {
'recall': 0.98, # Catch 98% of actual faces
'precision': 0.75, # 25% false alarms
'result': 'Many false alarms, but catches threats'
}
authentication_high_precision = {
'recall': 0.85, # Miss 15% of faces
'precision': 0.99, # Only 1% false positives
'result': 'Fewer false unlocks, some failed authentications'
}
# Tuning decision:
# Security system: Prefer high recall (catch all threats, review false alarms)
# Authentication: Prefer high precision (avoid false unlocks, user retries OK)
# Photo organization: Balance both (some misses OK, some false tags OK)False Positives vs False Negatives Impact#
False Positives (Detecting faces that aren’t there):
# Security system false positive analysis
daily_camera_triggers = 1000
false_positive_rate = 0.15 # 15% false alarms
# Operational impact:
false_alarms_per_day = daily_camera_triggers * false_positive_rate # 150
guard_time_per_review = 30 # seconds
daily_wasted_time = 150 * 30 / 3600 # 1.25 hours per day
# Annual cost:
security_guard_hourly = 25
annual_false_alarm_cost = 1.25 * 365 * security_guard_hourly # $11,406
# Plus: Alert fatigue, missed real threats, system distrust
# Mitigation strategies:
# - Higher detection threshold (fewer detections, fewer false positives)
# - Two-stage verification (fast detector + accurate validator)
# - Confidence scoring (only alert on high-confidence detections)False Negatives (Missing actual faces):
# Attendance system false negative analysis
students_per_day = 500
false_negative_rate = 0.08 # 8% missed detections
# Operational impact:
missed_attendance_per_day = students_per_day * false_negative_rate # 40 students
manual_correction_time = 120 # 2 minutes per correction
daily_admin_time = 40 * 120 / 3600 # 1.33 hours per day
# System reliability impact:
students_requiring_manual_entry = 40
system_trust_degradation = 0.4 # 40% less trust in automated system
manual_sign_in_adoption = 0.6 # 60% switch to manual process
# Result: Automated system becomes obsolete, wasted investment
# Critical applications where false negatives are costly:
# - Security access: Legitimate users locked out
# - Photo tagging: Memories/people missing from albums
# - Surveillance: Threats not detected
# - Medical: Patients not identifiedLatency Impact on Real-Time Applications#
Frame Rate Requirements by Application:
# Different applications have different latency budgets
application_fps_requirements = {
'security_monitoring': {
'fps': 10, # 100ms budget per frame
'reason': 'Motion detection, not smooth playback',
'acceptable_models': ['Haar', 'HOG', 'MTCNN', 'MediaPipe']
},
'video_conferencing': {
'fps': 30, # 33ms budget per frame
'reason': 'Smooth video for professional quality',
'acceptable_models': ['Haar', 'HOG', 'MediaPipe']
},
'ar_filters': {
'fps': 60, # 16ms budget per frame
'reason': 'High frame rate needed for immersion',
'acceptable_models': ['MediaPipe (barely)']
},
'photo_processing': {
'fps': 'N/A', # Batch processing
'reason': 'User waits for results, accuracy matters',
'acceptable_models': ['All models, prefer accuracy']
}
}
# Real-world latency measurements:
latency_breakdown_retinaface = {
'image_preprocessing': 5_ms,
'model_inference': 65_ms,
'postprocessing': 8_ms,
'total': 78_ms # 12 FPS - UNUSABLE for real-time
}
latency_breakdown_mediapipe = {
'image_preprocessing': 2_ms,
'model_inference': 8_ms,
'postprocessing': 2_ms,
'total': 12_ms # 83 FPS - excellent for real-time
}Dropped Frames and User Experience:
# Video call quality degradation
target_fps = 30
camera_capture_time = 1 # 1ms to capture frame
detection_time = 85 # RetinaFace: 85ms
# Frame processing time:
total_frame_time = camera_capture_time + detection_time # 86ms
achievable_fps = 1000 / total_frame_time # 11.6 FPS
# Frames dropped:
frames_dropped = target_fps - achievable_fps # 18.4 FPS dropped
frame_drop_percentage = frames_dropped / target_fps # 61% of frames dropped
# User experience impact:
# 30 FPS: Imperceptible lag, professional quality
# 15 FPS: Noticeable stutter, acceptable for casual use
# 11 FPS: Obvious lag, poor quality, user complaints
# <10 FPS: Unusable for live interaction
# Business consequences:
users_affected = 10_000
churn_rate_poor_quality = 0.35
monthly_revenue_per_user = 15
monthly_churn_cost = users_affected * churn_rate_poor_quality * monthly_revenue_per_user
# = $52,500 monthly recurring revenue lostResource Requirements#
CPU Requirements:
# CPU-only deployment costs
inference_models = {
'Haar Cascade': {
'cpu_cores': 1,
'cpu_utilization': 0.15, # 15% of one core
'hourly_cost': 0.02, # t3.small
'throughput_fps': 125
},
'Dlib HOG': {
'cpu_cores': 1,
'cpu_utilization': 0.35, # 35% of one core
'hourly_cost': 0.02, # t3.small
'throughput_fps': 66
},
'MTCNN': {
'cpu_cores': 2,
'cpu_utilization': 0.80, # 80% of two cores
'hourly_cost': 0.08, # t3.large
'throughput_fps': 22
},
'RetinaFace': {
'cpu_cores': 4,
'cpu_utilization': 1.00, # 100% of four cores
'hourly_cost': 0.16, # t3.xlarge
'throughput_fps': 1.2
}
}
# Monthly cost for 24/7 operation:
# Haar: $14.40/month - economical for continuous operation
# RetinaFace (CPU): $115.20/month - 8x more expensiveGPU Requirements:
# GPU acceleration benefits
gpu_acceleration = {
'RetinaFace': {
'cpu_time': 850_ms,
'gpu_time': 65_ms,
'speedup': 13.1, # 13x faster on GPU
'gpu_cost': 0.526, # per hour (g4dn.xlarge)
'monthly_cost': 378.72
},
'MTCNN': {
'cpu_time': 45_ms,
'gpu_time': 18_ms,
'speedup': 2.5, # 2.5x faster on GPU
'gpu_cost': 0.526,
'monthly_cost': 378.72
}
}
# When GPU acceleration is worth it:
# - High-accuracy models (RetinaFace) need GPU for reasonable speed
# - High volume processing (>10 detections/second sustained)
# - Real-time requirements with accurate models
# - Batch processing large photo collections
# When to avoid GPU:
# - Low volume (<1000 detections/day)
# - Fast models (Haar, HOG) already meet requirements on CPU
# - Cost-sensitive applications
# - Edge/embedded deployment (no GPU available)Mobile Device Requirements:
# Mobile deployment constraints
mobile_constraints = {
'model_size_limit': 10_MB, # User acceptance threshold
'inference_time_target': 33_ms, # 30 FPS requirement
'battery_drain_acceptable': 500_mW, # Sustainable usage
'memory_limit': 100_MB # Avoid app kills
}
# Model comparisons for mobile:
mobile_models = {
'MediaPipe': {
'model_size': 3.2_MB, # ✓ Under threshold
'inference_time': 12_ms, # ✓ Real-time capable
'power_consumption': 450_mW, # ✓ Efficient
'memory_usage': 45_MB, # ✓ Low memory
'verdict': 'Ideal for mobile'
},
'Dlib CNN': {
'model_size': 10.5_MB, # ✗ At threshold limit
'inference_time': 95_ms, # ✗ Too slow (10 FPS)
'power_consumption': 1800_mW, # ✗ Drains battery
'memory_usage': 180_MB, # ✗ High memory
'verdict': 'Avoid for mobile'
},
'RetinaFace': {
'model_size': 26.8_MB, # ✗ Far too large
'inference_time': 340_ms, # ✗ Unusable (3 FPS)
'power_consumption': 3200_mW, # ✗ Severe battery drain
'memory_usage': 450_MB, # ✗ Memory pressure
'verdict': 'Impossible for mobile'
}
}
# Mobile app download conversion rates:
app_size_conversion = {
'under_50mb': 0.85, # 85% conversion
'under_100mb': 0.68, # 68% conversion
'100mb_plus': 0.51 # 51% conversion
}
# Model choice affects app size affects downloads affects revenueKey Technical Concepts#
Facial Landmarks: Why Point Count Matters#
5-Point Landmarks (Minimal Detection):
# Basic landmark positions
landmarks_5 = {
'left_eye': (x1, y1),
'right_eye': (x2, y2),
'nose_tip': (x3, y3),
'left_mouth_corner': (x4, y4),
'right_mouth_corner': (x5, y5)
}
# What you can do with 5 points:
capabilities_5_point = [
'Face alignment (rotation correction)',
'Eye detection (blink detection, gaze estimation)',
'Basic face recognition (alignment for embedding)',
'Simple crop/zoom (center on eyes)',
]
# Use cases:
# - Face recognition systems (minimal alignment needed)
# - Photo cropping/rotation
# - Basic driver drowsiness detection
# - Login/authentication systems
# Performance:
detection_time_5_point = 3_ms # Very fast
accuracy = 0.95 # Robust for these key points
model_size = 1_MB # Tiny model68-Point Landmarks (Standard Detection):
# Comprehensive facial features
landmarks_68 = {
'jaw_contour': 17, # Face outline
'eyebrows': 10, # Left and right eyebrows
'nose_bridge': 4, # Top of nose
'nose_bottom': 5, # Nostrils and nose tip
'eyes': 12, # Eye contours (6 points each)
'mouth_outer': 12, # Outer lip contour
'mouth_inner': 8 # Inner lip contour
}
# What you can do with 68 points:
capabilities_68_point = [
'Emotion detection (mouth shape, eyebrow position)',
'Face morphing (detailed feature manipulation)',
'Makeup application (precise feature boundaries)',
'Face swapping (accurate feature alignment)',
'Detailed animation (avatar facial expressions)',
'Medical analysis (facial symmetry assessment)'
]
# Use cases:
# - Social media filters (beautification, face swap)
# - Emotion recognition systems
# - Medical/psychological research
# - Character animation
# - Video conferencing effects
# Performance:
detection_time_68_point = 15_ms # Still real-time capable
accuracy = 0.88 # More points = harder to be precise
model_size = 8_MB # Larger model468-Point 3D Mesh (MediaPipe Face Mesh):
# Full 3D face reconstruction
mesh_468_points = {
'total_points': 468,
'geometry': '3D coordinates (x, y, z)',
'coverage': 'Complete face surface mesh',
'includes': 'Face contour, eyes, eyebrows, nose, mouth, face interior'
}
# What you can do with 468-point 3D mesh:
capabilities_468_mesh = [
'Advanced AR effects (3D objects on face)',
'Realistic face tracking (depth-aware)',
'Head pose estimation (3D orientation)',
'Lighting-aware effects (surface normals)',
'Occlusion handling (which objects go behind face)',
'Realistic face filters (depth-based blur)',
'3D avatar animation (full face capture)'
]
# Use cases:
# - Snapchat/Instagram AR filters
# - Virtual try-on (glasses, makeup, accessories)
# - VR/AR applications
# - Motion capture for animation
# - Virtual avatar control
# - Face-based game controls
# Performance:
detection_time_468_mesh = 25_ms # Still usable for real-time (40 FPS)
accuracy_2d = 0.92 # Good 2D accuracy
accuracy_3d = 0.85 # Approximate 3D reconstruction
model_size = 3.2_MB # Optimized for mobile
# 3D depth accuracy:
# Relative depth: Excellent (which features are closer/farther)
# Absolute depth: Approximate (estimated from single camera)
# Sufficient for: AR effects, not for precise 3D scanningLandmark Count Decision Matrix:
def choose_landmark_model(use_case):
decision_tree = {
'face_recognition': '5-point', # Just need alignment
'photo_cropping': '5-point', # Basic positioning
'emotion_detection': '68-point', # Need expression details
'ar_filters_2d': '68-point', # Need feature boundaries
'ar_filters_3d': '468-point', # Need 3D mesh
'makeup_application': '68-point', # Feature boundaries
'face_swap': '68-point', # Detailed alignment
'avatar_control': '468-point', # Full face capture
'virtual_try_on': '468-point' # 3D understanding
}
return decision_tree.get(use_case)
# Performance vs capability trade-off:
# More landmarks = More capabilities BUT slower + less accurate
# Only use detailed landmarks if you actually need themFace Recognition vs Detection vs Landmarks#
Critical Distinction:
# Three separate tasks, often confused:
# 1. FACE DETECTION: "Is there a face? Where?"
face_detection = {
'input': 'Image',
'output': 'Bounding boxes [(x, y, w, h), ...]',
'question': 'Where are the faces in this image?',
'complexity': 'Low',
'speed': 'Fast (5-100ms)',
'use_cases': ['Count faces', 'Crop to face', 'Focus camera']
}
# 2. FACIAL LANDMARKS: "Where are the features?"
facial_landmarks = {
'input': 'Face bounding box',
'output': 'Feature points [(x1,y1), (x2,y2), ...]',
'question': 'Where are the eyes, nose, mouth?',
'complexity': 'Medium',
'speed': 'Medium (10-50ms)',
'use_cases': ['Face alignment', 'Expression analysis', 'AR filters']
}
# 3. FACE RECOGNITION: "Whose face is it?"
face_recognition = {
'input': 'Face image (aligned)',
'output': 'Identity vector (embedding)',
'question': 'Which person is this?',
'complexity': 'High',
'speed': 'Slow (50-200ms)',
'use_cases': ['Login', 'Photo tagging', 'Security access']
}
# Pipeline dependencies:
# Recognition requires: Detection → Alignment (landmarks) → Recognition
# Each step adds latency and potential for errorsWhy This Matters for Library Selection:
# Example: Photo album organization
# WRONG APPROACH (conflating tasks):
# "I need face detection to organize my photos by person"
# Reality: You need detection + landmarks + recognition
# Single library may not do all three well
# RIGHT APPROACH (separate concerns):
pipeline = {
'detection': 'MediaPipe BlazeFace', # Fast detection: 6ms
'landmarks': 'MediaPipe Face Mesh', # Fast landmarks: 12ms
'recognition': 'FaceNet/ArcFace', # Accurate identity: 50ms
'total_time': 68_ms # 14 FPS, acceptable for batch
}
# Library capabilities matrix:
library_capabilities = {
'OpenCV Haar': {
'detection': True, # ✓ Basic detection
'landmarks': False, # ✗ No landmarks
'recognition': False # ✗ No recognition
},
'Dlib': {
'detection': True, # ✓ HOG detector
'landmarks': True, # ✓ 5 or 68 points
'recognition': True # ✓ ResNet embeddings
},
'MediaPipe': {
'detection': True, # ✓ BlazeFace
'landmarks': True, # ✓ 468-point mesh
'recognition': False # ✗ No identity (detection only)
},
'InsightFace': {
'detection': True, # ✓ RetinaFace
'landmarks': True, # ✓ 5-point alignment
'recognition': True # ✓ ArcFace embeddings
}
}
# Choosing libraries based on actual needs:
if only_need_detection:
use('OpenCV Haar') # Fastest, simplest
elif need_detection_and_ar:
use('MediaPipe') # Detection + 3D mesh
elif need_full_recognition_pipeline:
use('InsightFace') # All-in-one solution
elif need_best_of_each:
use_combination([
('MediaPipe', 'detection'),
('Dlib', 'landmarks'),
('FaceNet', 'recognition')
])3D Face Modeling: When and Why#
2D Landmarks vs 3D Mesh:
# 2D Landmarks (traditional):
landmarks_2d = {
'coordinates': '(x, y)', # Pixel positions only
'depth_info': None, # No depth information
'head_rotation': 'estimated', # Inferred from point positions
'use_cases': [
'Face alignment',
'2D filters (sunglasses, mustache)',
'Emotion detection',
'Basic AR effects'
]
}
# 3D Mesh (modern):
mesh_3d = {
'coordinates': '(x, y, z)', # Includes depth
'depth_info': 'per-vertex', # Each point has depth
'head_rotation': 'calculated', # Precise 3D orientation
'surface_normals': True, # Lighting direction per triangle
'use_cases': [
'3D AR effects (objects behind/in-front of face)',
'Realistic lighting on virtual objects',
'Depth-aware blur/focus',
'Accurate occlusion handling',
'VR avatar animation',
'Virtual makeup with proper shading'
]
}
# Concrete example: Virtual sunglasses
# 2D approach:
# - Detect eyes, place sunglasses image at eye position
# - Problem: Sunglasses don't rotate with head
# - Problem: No depth, looks "pasted on"
# Result: Obviously fake, breaks immersion
# 3D approach:
# - Track 3D face mesh, calculate head rotation
# - Place 3D sunglasses model on face in 3D space
# - Render with proper perspective and lighting
# Result: Realistic tracking, looks naturalDepth Estimation from Single Image:
# How MediaPipe estimates 3D from 2D camera:
depth_estimation_approach = {
'training': 'Trained on 3D face scans + 2D images',
'method': 'Neural network predicts z-coordinate from x,y',
'accuracy': 'Relative depth accurate, absolute depth approximate',
'limitations': [
'Scale ambiguous (big face far away = small face close up)',
'Depth estimate, not measurement',
'Works for faces, not general 3D scanning'
]
}
# Accuracy of depth estimation:
relative_depth_accuracy = 0.90 # Very good at "nose is closer than ears"
absolute_depth_accuracy = 0.60 # Less accurate at "face is 50cm away"
# Sufficient for AR effects:
ar_requirements = {
'occlusion_order': 'Relative depth only', # ✓ Excellent
'lighting_effects': 'Surface normals', # ✓ Good
'object_placement': 'Relative positioning', # ✓ Good
'3d_measurement': 'Absolute depth', # ✗ Insufficient
}
# Example: Virtual hat placement
virtual_hat = {
'needs_relative_depth': True, # Hat on top of head, not behind
'needs_absolute_depth': False, # Don't care exact cm from camera
'mediapipe_suitable': True # ✓ Perfect for this use case
}
# Example: Face measurement for glasses sizing
glasses_sizing = {
'needs_relative_depth': False,
'needs_absolute_depth': True, # Need actual face width in cm
'mediapipe_suitable': False, # ✗ Need stereo camera or depth sensor
}Why 3D Matters for Specific Applications:
# Application 1: AR Makeup Application
ar_makeup_2d = {
'approach': 'Overlay lipstick color on lip region',
'problem': 'Flat overlay, no shading',
'realism': 'Low - obviously computer-generated',
'user_satisfaction': 0.65
}
ar_makeup_3d = {
'approach': 'Apply color with lighting based on 3D mesh normals',
'benefit': 'Natural shading, follows lip curves',
'realism': 'High - looks like real makeup',
'user_satisfaction': 0.89,
'conversion_lift': 1.37 # 37% more likely to purchase
}
# Business impact:
monthly_users = 50_000
purchase_conversion_2d = 0.03 # 3% conversion with flat overlay
purchase_conversion_3d = 0.041 # 4.1% with realistic 3D shading
average_order_value = 45
revenue_2d = monthly_users * purchase_conversion_2d * average_order_value
# = $67,500/month
revenue_3d = monthly_users * purchase_conversion_3d * average_order_value
# = $92,250/month
revenue_lift = revenue_3d - revenue_2d
# = $24,750/month additional revenue from 3D mesh
# Justifies higher development complexity
# Application 2: Head Pose Estimation
head_pose_2d = {
'accuracy': 'Approximate from landmark positions',
'yaw_accuracy': 15, # degrees error
'pitch_accuracy': 20, # degrees error
'roll_accuracy': 10, # degrees error
'use_cases': 'Basic gaze tracking, rough attention detection'
}
head_pose_3d = {
'accuracy': 'Calculated from 3D mesh orientation',
'yaw_accuracy': 3, # degrees error (5x better)
'pitch_accuracy': 4, # degrees error (5x better)
'roll_accuracy': 2, # degrees error (5x better)
'use_cases': 'Precise gaze tracking, driver attention, VR control'
}
# Application: Driver drowsiness detection
driver_monitoring_requirements = {
'head_pose_accuracy': '<5 degrees', # Safety critical
'2d_landmarks': 'Insufficient', # ✗ Too imprecise
'3d_mesh': 'Required', # ✓ Meets requirements
}Real-World Performance Patterns#
Security Camera System#
# 24/7 surveillance deployment
cameras = 50
resolution = (1920, 1080)
target_fps = 15 # Security doesn't need 30 FPS
recording_hours = 24
# Performance requirements:
detection_budget = 66_ms # 15 FPS = 66ms per frame
false_negative_tolerance = 0.05 # Max 5% missed detections (security critical)
false_positive_tolerance = 0.20 # 20% false alarms acceptable (reviewed by guards)
# Algorithm selection:
options = {
'Haar Cascade': {
'detection_time': 8_ms, # ✓ Within budget
'recall': 0.88, # ✗ 12% missed (above tolerance)
'precision': 0.82, # ✓ Within tolerance
'verdict': 'Too many missed detections for security'
},
'Dlib HOG': {
'detection_time': 15_ms, # ✓ Within budget
'recall': 0.94, # ✓ 6% missed (borderline acceptable)
'precision': 0.85, # ✓ Within tolerance
'verdict': 'Acceptable trade-off'
},
'MTCNN': {
'detection_time': 45_ms, # ✓ Within budget (66ms)
'recall': 0.97, # ✓ 3% missed (excellent)
'precision': 0.91, # ✓ High precision
'verdict': 'Best choice - high recall, within budget'
}
}
# Deployment costs (50 cameras):
mtcnn_cpu_cost = {
'server_needed': 'c5.4xlarge', # 16 vCPU for 50 camera streams
'hourly_cost': 0.68,
'monthly_cost': 489.60,
'cost_per_camera': 9.79 # $9.79/camera/month
}
# Operational cost comparison:
human_monitoring_cost = {
'guards_needed': 6, # Round-the-clock coverage
'hourly_wage': 25,
'monthly_cost': 108_000, # $108k/month
'cost_per_camera': 2160 # $2,160/camera/month
}
# ROI: Automated detection = 0.45% of human monitoring cost
# Even expensive detection algorithms are economical vs manual monitoringBatch Photo Processing#
# Photo album organization system
total_photos = 50_000
photos_with_faces = 35_000 # 70% contain faces
average_faces_per_photo = 2.3
# Processing requirements:
total_detections = photos_with_faces * average_faces_per_photo # 80,500 detections
acceptable_processing_time = 8 # hours (overnight batch job)
acceptable_time_per_photo = (8 * 3600) / total_photos # 576ms per photo
# Algorithm selection for accuracy:
processing_options = {
'Haar Cascade': {
'time_per_photo': 25_ms,
'total_time': 0.35, # hours - very fast
'accuracy': 0.85, # Misses 15% of faces
'missed_faces': 12_075, # 12k faces not tagged
'verdict': 'Too fast but inaccurate - poor user experience'
},
'RetinaFace': {
'time_per_photo': 120_ms,
'total_time': 1.67, # hours - within budget
'accuracy': 0.98, # Misses 2% of faces
'missed_faces': 1_610, # 1.6k faces not tagged
'verdict': 'Optimal - accurate, completes overnight'
}
}
# User experience impact:
# Album with 1000 photos, 1800 faces:
user_album_haar = {
'faces_detected': 1530, # 85% recall
'faces_missed': 270, # 15% missed
'user_experience': 'Frustrated - many people not found'
}
user_album_retinaface = {
'faces_detected': 1764, # 98% recall
'faces_missed': 36, # 2% missed
'user_experience': 'Satisfied - nearly all people found'
}
# Cost comparison:
retinaface_gpu_cost = {
'instance': 'g4dn.xlarge',
'hourly_cost': 0.526,
'processing_time': 1.67, # hours
'total_cost': 0.88, # $0.88 per user album
'cost_per_photo': 0.0000176 # $0.000018 per photo
}
# Batch processing optimization:
# GPU utilization: Process 4 images in parallel
# Actual processing time: 1.67 / 4 = 0.42 hours
# Actual cost per album: $0.22
# Very affordable for high-quality resultsReal-Time Video Conferencing#
# Video call background blur/effects
resolution = (1280, 720)
target_fps = 30
users_concurrent = 5_000
# Hard constraints:
frame_budget = 33_ms # Must maintain 30 FPS
detection_budget = 15_ms # Detection + blur processing
acceptable_cpu = 40 # % CPU usage (leave room for other tasks)
# Algorithm comparison:
video_conference_options = {
'Haar Cascade': {
'detection_time': 5_ms, # ✓ Well within budget
'cpu_usage': 18, # ✓ Low CPU usage
'accuracy': 0.85, # ✗ Occasional face misdetection
'edge_quality': 'rough', # Rough bounding box
'verdict': 'Fast but rough - visible artifacts'
},
'MediaPipe': {
'detection_time': 12_ms, # ✓ Within budget
'cpu_usage': 35, # ✓ Acceptable CPU
'accuracy': 0.94, # ✓ Reliable detection
'edge_quality': 'precise', # 468-point mesh = accurate edges
'verdict': 'Optimal - smooth edges, reliable, efficient'
},
'RetinaFace': {
'detection_time': 85_ms, # ✗ Exceeds budget by 5.7x
'cpu_usage': 95, # ✗ Maxes out CPU
'accuracy': 0.98, # ✓ High accuracy (wasted)
'edge_quality': 'very precise',
'verdict': 'Too slow - drops frames, poor experience'
}
}
# Frame drop calculation:
retinaface_frame_time = 85_ms
target_frame_time = 33_ms
frames_processed = 1000 / 85 # 11.7 FPS
frames_dropped = 30 - 11.7 # 18.3 FPS dropped
drop_percentage = 61 # 61% of frames dropped
# User experience impact:
user_experience_scores = {
'30_fps': {
'smoothness': 0.95,
'professional_quality': True,
'churn_rate': 0.08
},
'15_fps': {
'smoothness': 0.72,
'professional_quality': False,
'churn_rate': 0.22
},
'11_fps': {
'smoothness': 0.45,
'professional_quality': False,
'churn_rate': 0.41
}
}
# Business impact (5000 concurrent users):
monthly_subscription = 15
churn_30fps = 5000 * 0.08 * 15 # $6,000/month churn
churn_11fps = 5000 * 0.41 * 15 # $30,750/month churn
# Wrong algorithm = $24,750/month additional churnMobile AR Filters Application#
# Snapchat/Instagram-style face filters
target_devices = ['iPhone 12', 'Samsung Galaxy S21', 'Budget Android']
target_fps = 30 # Minimum for smooth AR
battery_life_target = 120 # Minutes of continuous use
# Mobile-specific constraints:
constraints = {
'app_size_limit': 50_MB, # Above this, download conversion drops
'model_size_budget': 10_MB, # Max for face detection model
'inference_time': 33_ms, # 30 FPS requirement
'power_consumption': 500_mW, # Sustainable battery drain
'memory_limit': 100_MB # OS kills apps above this
}
# Algorithm comparison on iPhone 12:
iphone_performance = {
'Dlib CNN': {
'model_size': 10.5_MB, # ✗ At limit, adds to app size
'inference_time': 95_ms, # ✗ Too slow (10 FPS)
'power': 1800_mW, # ✗ Heavy battery drain
'battery_life': 35, # minutes - unusable
'landmarks': 68, # Not 3D (insufficient for AR)
'verdict': 'Unusable for mobile AR'
},
'MediaPipe Face Mesh': {
'model_size': 3.2_MB, # ✓ Small, minimal app size impact
'inference_time': 12_ms, # ✓ Excellent (83 FPS capable)
'power': 450_mW, # ✓ Efficient
'battery_life': 115, # minutes - acceptable
'landmarks': 468, # 3D mesh (perfect for AR)
'verdict': 'Designed for this use case'
}
}
# Budget Android device (weaker hardware):
budget_android_performance = {
'MediaPipe Face Mesh': {
'inference_time': 28_ms, # ✓ Still real-time (35 FPS)
'power': 680_mW, # Higher power, but acceptable
'battery_life': 85, # minutes - acceptable
'verdict': 'Works across device range'
}
}
# App store conversion rates:
app_store_metrics = {
'under_50mb': {
'download_conversion': 0.85,
'wifi_only_downloads': 0.30 # 30% wait for WiFi
},
'over_50mb': {
'download_conversion': 0.51, # 40% drop
'wifi_only_downloads': 0.75 # 75% wait for WiFi
}
}
# Business impact (1M impressions/month):
app_impressions = 1_000_000
small_app_installs = app_impressions * 0.85 # 850k installs
large_app_installs = app_impressions * 0.51 # 510k installs
lost_installs = 340_000 # From model size choice
# Monetization impact:
ad_revenue_per_user = 0.25 # Monthly ad revenue
lost_monthly_revenue = lost_installs * ad_revenue_per_user
# = $85,000/month lost from wrong model choiceAttendance/Access Control System#
# Classroom/office attendance tracking
daily_check_ins = 500
processing_time_per_person = 3 # seconds at terminal
peak_hour_traffic = 200 # People between 8-9 AM
# System requirements:
detection_accuracy_required = 0.98 # High accuracy (attendance records)
processing_time_budget = 2 # seconds (faster than manual)
false_negative_tolerance = 0.02 # Max 2% missed (critical for attendance)
# Algorithm selection:
attendance_options = {
'Haar Cascade': {
'detection_time': 0.008, # seconds - very fast
'recognition_pipeline': 0.05, # Total time with recognition
'accuracy': 0.92, # ✗ 8% miss rate too high
'false_negatives': 40, # 40 people missed per day
'verdict': 'Too inaccurate for attendance'
},
'MTCNN + ArcFace': {
'detection_time': 0.045, # seconds
'recognition_pipeline': 0.15, # Total time
'accuracy': 0.98, # ✓ Meets requirements
'false_negatives': 10, # 10 people missed per day
'verdict': 'Accurate, within time budget'
}
}
# Operational impact of false negatives:
missed_attendance_daily = 10
correction_time_per_case = 120 # 2 minutes manual correction
daily_admin_burden = 10 * 120 / 60 # 20 minutes per day
# Student/employee experience:
student_impact = {
'false_negative': 'Marked absent, must appeal manually',
'appeal_time': 15, # minutes per appeal
'frustration_level': 'high',
'system_trust': 'low'
}
# Peak hour throughput:
processing_rate_mtcnn = 1 / 0.15 # 6.7 people per minute
peak_hour_capacity = 6.7 * 60 # 402 people per hour
peak_requirement = 200 # ✓ Sufficient capacity
# Single point of failure mitigation:
terminals_needed = {
'fast_model': 1, # 6.7/min handles 200/hr easily
'slow_model': 2, # Need backup for reliability
}
# Hardware costs:
terminal_cost = 800 # Tablet + camera + mount
mtcnn_compute = 'edge', # Can run on device
monthly_cloud_cost = 0 # No cloud inference needed
# One-time hardware cost, no ongoing cloud costsCommon Pitfalls#
Pitfall 1: Using High-Accuracy Model for Real-Time#
# Mistake: Choosing RetinaFace for video conferencing
implementation_mistake = {
'chosen_model': 'RetinaFace',
'reason': 'Highest accuracy on benchmarks',
'detection_time': 85_ms,
'target_fps': 30,
'result': 'Dropped frames, choppy video'
}
# What actually happens:
frame_time_available = 33_ms
detection_takes = 85_ms
frames_behind = 85 / 33 # 2.6 frames behind
video_lag = 'Noticeable, unusable'
# User complaints:
complaints = [
'Video is laggy',
'Audio/video out of sync',
'Background blur flickers',
'Effects don't track well'
]
# Fix: Match model to latency requirements
correct_implementation = {
'chosen_model': 'MediaPipe',
'reason': 'Real-time performance + acceptable accuracy',
'detection_time': 12_ms,
'achievable_fps': 83,
'result': 'Smooth video, happy users'
}
# Key lesson:
# Benchmarks show accuracy, not real-world suitability
# "Best" accuracy doesn't mean "best" for your use casePitfall 2: Using Fast Model for Difficult Conditions#
# Mistake: Haar Cascade for outdoor security cameras
implementation_mistake = {
'chosen_model': 'Haar Cascade',
'reason': 'Fast, low cost',
'conditions': 'Variable lighting, distances, angles',
'accuracy_in_practice': 0.68 # Much lower than lab testing
}
# Why accuracy drops in production:
production_challenges = {
'lab_testing': {
'lighting': 'Consistent',
'face_size': 'Optimal',
'angle': 'Frontal',
'occlusion': 'None',
'accuracy': 0.85
},
'production': {
'lighting': 'Variable (day/night, shadows)',
'face_size': 'Small to large',
'angle': 'All angles',
'occlusion': 'Hats, glasses, masks',
'accuracy': 0.68 # 20% drop
}
}
# Security system failure:
missed_detections_per_day = 32 # 32% miss rate
security_incidents_detected = 0.68 # Only catch 68% of incidents
system_reliability = 'Unacceptable for security'
# Fix: Match model capability to conditions
correct_implementation = {
'chosen_model': 'MTCNN',
'handles_difficult_conditions': True,
'accuracy_in_production': 0.94,
'additional_cost': '$10/camera/month',
'result': 'Reliable security monitoring'
}
# Cost of failure:
single_security_incident_cost = 50_000 # Average cost
probability_of_incident = 0.02 # 2% per year
expected_annual_cost = 50_000 * 0.02 # $1,000
# Spending $120/year extra per camera to prevent $1,000 loss = good ROIPitfall 3: Not Considering Lighting Conditions#
# Mistake: Indoor-tested model for outdoor deployment
implementation_mistake = {
'testing_environment': 'Indoor, controlled lighting',
'test_accuracy': 0.96,
'deployment_environment': 'Outdoor, variable lighting',
'actual_accuracy': 0.72 # 25% drop
}
# Why lighting matters:
lighting_impact = {
'frontal_lighting': {
'accuracy': 0.96, # Optimal
'conditions': 'Indoor, studio, ideal'
},
'side_lighting': {
'accuracy': 0.88, # -8%
'conditions': 'Half face in shadow'
},
'backlighting': {
'accuracy': 0.65, # -31%
'conditions': 'Face dark, background bright'
},
'low_light': {
'accuracy': 0.58, # -40%
'conditions': 'Night, minimal lighting'
}
}
# Real-world scenario: Outdoor event attendance
outdoor_event = {
'time': 'All day (morning to evening)',
'weather': 'Variable',
'lighting_conditions': [
'Morning: low angle sun (backlighting)',
'Noon: harsh overhead sun (shadows)',
'Afternoon: side lighting',
'Evening: low light'
],
'attendance_accuracy_basic_model': 0.68,
'attendance_accuracy_robust_model': 0.91
}
# Operational impact:
event_attendees = 1000
basic_model_misses = 1000 * 0.32 # 320 missed
robust_model_misses = 1000 * 0.09 # 90 missed
manual_corrections_needed = 320 # vs 90
# Manual correction cost:
correction_time = 2 # minutes each
total_correction_time = 320 * 2 # 640 minutes = 10.7 hours
staff_hourly_cost = 25
correction_cost = 10.7 * 25 # $267.50 per event
# Model upgrade cost:
better_model_monthly = 50
events_per_month = 4
cost_per_event = 12.50 # $12.50 per event
# Saving $255 per event by using better modelPitfall 4: Ignoring Face Size Constraints#
# Mistake: Not testing with actual face sizes in your application
implementation_mistake = {
'testing': 'Large, close-up faces',
'test_accuracy': 0.94,
'production': 'Small, distant faces',
'actual_accuracy': 0.56 # Dramatic drop
}
# Face size impact on detection:
face_size_accuracy = {
'large_faces': {
'size': '>200px',
'percentage_of_image': '>30%',
'accuracy': 0.95,
'all_models_work': True
},
'medium_faces': {
'size': '80-200px',
'percentage_of_image': '10-30%',
'accuracy': 0.85,
'fast_models_struggle': True
},
'small_faces': {
'size': '30-80px',
'percentage_of_image': '3-10%',
'accuracy': 0.65,
'need_specialized_models': True
},
'tiny_faces': {
'size': '<30px',
'percentage_of_image': '<3%',
'accuracy': 0.35,
'most_models_fail': True
}
}
# Real-world scenario: Classroom monitoring
classroom_camera = {
'camera_resolution': (1920, 1080),
'room_size': '30 feet',
'students': 30,
'face_sizes': '40-80px', # Small faces
'haar_cascade_accuracy': 0.48, # Fails
'retinaface_accuracy': 0.89 # Much better
}
# Multi-scale detection strategy:
multi_scale_approach = {
'pyramid_levels': 5, # Process image at multiple scales
'scales': [1.0, 0.75, 0.5, 0.25, 0.125],
'processing_time': '3-5x slower',
'small_face_accuracy': '+40%', # Significant improvement
'when_to_use': 'Variable face sizes (crowds, surveillance)'
}
# Testing checklist:
face_size_testing = [
'Measure actual face sizes in production images',
'Test model at those specific sizes',
'Consider multi-scale if sizes vary >2x',
'Benchmark accuracy per size range',
'Set minimum detectable size expectations'
]Pitfall 5: Mobile Deployment Without Optimization#
# Mistake: Using desktop model directly on mobile
implementation_mistake = {
'model': 'Dlib CNN (desktop version)',
'model_size': 10.5_MB,
'inference_time': 340_ms, # On mobile CPU
'power_consumption': 3200_mW,
'battery_drain': 'Severe'
}
# User experience:
mobile_problems = {
'battery_life': '25 minutes continuous use',
'phone_heating': 'Device becomes hot',
'throttling': 'CPU throttles, performance degrades',
'app_crashes': 'iOS kills app for excessive resources',
'user_reviews': '1-2 stars, "drains battery"'
}
# Proper mobile optimization:
optimization_techniques = {
'model_quantization': {
'float32_to_int8': True,
'size_reduction': '75%', # 10.5MB → 2.6MB
'speed_improvement': '2-3x',
'accuracy_loss': '1-2%' # Acceptable
},
'mobile_architecture': {
'use': 'MobileNet, EfficientNet backbones',
'designed_for': 'Mobile hardware',
'benefits': '5-10x faster on mobile'
},
'inference_optimization': {
'framework': 'TensorFlow Lite, Core ML',
'hardware_acceleration': 'Neural Engine (iOS), GPU',
'speed_improvement': '3-5x'
}
}
# Optimized mobile deployment:
optimized_mobile = {
'model': 'MediaPipe (mobile-optimized)',
'model_size': 3.2_MB,
'inference_time': 12_ms,
'power_consumption': 450_mW,
'battery_life': '120 minutes',
'user_reviews': '4.5 stars'
}
# App success comparison:
app_metrics = {
'unoptimized': {
'downloads': 100_000,
'active_users_30d': 15_000, # 15% retention
'avg_session_time': 3, # minutes
'rating': 2.1
},
'optimized': {
'downloads': 100_000,
'active_users_30d': 68_000, # 68% retention
'avg_session_time': 18, # minutes
'rating': 4.4
}
}
# Revenue impact:
ad_revenue_per_user = 0.25 # Monthly
unoptimized_revenue = 15_000 * 0.25 # $3,750/month
optimized_revenue = 68_000 * 0.25 # $17,000/month
# Proper mobile optimization = 4.5x revenue increasePerformance Optimization Strategies#
1. Cascade Detection for Efficiency#
# Two-stage detection: Fast filter + Accurate validator
cascade_approach = {
'stage_1': {
'model': 'Haar Cascade',
'purpose': 'Eliminate obvious non-faces',
'speed': 5_ms,
'recall': 0.99, # Catch almost all faces
'precision': 0.65, # Many false positives OK
'eliminates': '95% of image regions'
},
'stage_2': {
'model': 'RetinaFace',
'purpose': 'Verify detected regions',
'speed': 8_ms, # Only on candidate regions
'recall': 0.98, # Accurate final detection
'precision': 0.97, # High precision
'processes': '5% of image regions'
}
}
# Performance calculation:
single_stage_retinaface = {
'full_image_processing': 85_ms,
'throughput': 11.7 # FPS
}
cascade_retinaface = {
'stage1_time': 5_ms,
'stage2_time': 8_ms, # Only on 5% of regions
'total_time': 13_ms,
'throughput': 76.9, # FPS
'speedup': 6.5 # 6.5x faster
}
# Accuracy comparison:
accuracy_comparison = {
'single_stage': 0.98, # High accuracy
'cascade': 0.97, # 1% drop
'trade_off': 'Acceptable - 6.5x speed for 1% accuracy'
}
# When to use cascade:
cascade_use_cases = [
'Need high accuracy but also real-time performance',
'Large images with few faces',
'Video streams where most frames similar',
'Batch processing where speed matters'
]
# Implementation:
def cascade_detection(image):
# Stage 1: Fast, high recall
candidate_regions = haar_cascade.detect(image) # 5ms
# Stage 2: Accurate verification on candidates only
verified_faces = []
for region in candidate_regions:
crop = image[region]
if retinaface.verify(crop): # 8ms per region
verified_faces.append(region)
return verified_faces
# Total: 13ms vs 85ms for full RetinaFace2. Region of Interest Tracking#
# Optimization: Don't detect every frame in video
roi_tracking_approach = {
'frame_1': 'Full detection', # Expensive
'frames_2_10': 'Track existing faces', # Cheap
'frame_11': 'Full detection', # Re-detect periodically
'strategy': 'Detect every Nth frame, track between'
}
# Performance improvement:
full_detection_every_frame = {
'detection_time': 15_ms, # MTCNN per frame
'fps_budget': 30,
'total_detection_time_per_second': 450_ms, # 15ms * 30 frames
'cpu_usage': 45 # Percent
}
detection_plus_tracking = {
'detection_every': 10, # Frames
'detection_time': 15_ms, # Every 10th frame
'tracking_time': 2_ms, # Other 9 frames
'total_time_per_second': 48_ms, # (15ms + 9*2ms) * 3
'cpu_usage': 5, # Percent
'speedup': 9.4 # 9.4x reduction
}
# Tracking algorithms:
tracking_options = {
'optical_flow': {
'speed': 2_ms,
'accuracy': 'Good for small movements',
'limitations': 'Fails with fast motion'
},
'kalman_filter': {
'speed': 1_ms,
'accuracy': 'Smooth predictions',
'limitations': 'Assumes constant motion'
},
'correlation_filter': {
'speed': 3_ms,
'accuracy': 'Robust to appearance changes',
'limitations': 'Slight drift over time'
}
}
# Tracking accuracy degradation:
frames_since_detection = [1, 2, 3, 4, 5, 10, 20, 30]
tracking_accuracy = [0.99, 0.98, 0.97, 0.96, 0.95, 0.90, 0.82, 0.70]
# Re-detect when accuracy drops below threshold (e.g., every 10 frames)
# Implementation:
class VideoFaceDetector:
def __init__(self):
self.detector = MTCNN()
self.tracker = OpticalFlowTracker()
self.detect_interval = 10
self.frame_count = 0
def process_frame(self, frame):
self.frame_count += 1
if self.frame_count % self.detect_interval == 1:
# Full detection
faces = self.detector.detect(frame) # 15ms
self.tracker.init(faces)
else:
# Just track existing faces
faces = self.tracker.update(frame) # 2ms
return faces
# Result: 9x faster while maintaining accuracy3. Multi-Scale Detection Trade-offs#
# Image pyramid for detecting faces at different scales
multi_scale_strategy = {
'scales': [1.0, 0.75, 0.5, 0.25], # Process image at multiple sizes
'purpose': 'Detect both large and small faces',
'trade_off': 'Better detection but slower'
}
# Performance impact:
single_scale_detection = {
'scales_processed': 1,
'detection_time': 10_ms,
'faces_detected': 'Only medium-sized',
'miss_rate': 0.35 # Miss small/large faces
}
multi_scale_detection = {
'scales_processed': 4,
'detection_time': 35_ms, # 3.5x slower
'faces_detected': 'All sizes',
'miss_rate': 0.08 # Much better
}
# Adaptive multi-scale:
adaptive_strategy = {
'initial_scan': 'Multi-scale (identify size distribution)',
'subsequent_frames': 'Single scale (at dominant size)',
're_scan': 'Every 100 frames',
'benefit': 'First frame accuracy with later frame speed'
}
# Smart scale selection:
def adaptive_multi_scale(image, detected_faces_history):
if len(detected_faces_history) < 10:
# Not enough history, use multi-scale
scales = [1.0, 0.75, 0.5, 0.25]
else:
# Analyze face size distribution
avg_face_size = calculate_avg_size(detected_faces_history)
if avg_face_size > 150:
scales = [1.0, 0.75] # Large faces only
elif avg_face_size > 80:
scales = [1.0, 0.5] # Medium faces
else:
scales = [0.5, 0.25] # Small faces only
return detect_at_scales(image, scales)
# Result: 2x faster than full multi-scale while maintaining accuracy4. GPU Acceleration When Available#
# CPU vs GPU performance comparison
model_performance = {
'RetinaFace': {
'cpu_time': 850_ms,
'gpu_time': 65_ms,
'speedup': 13.1,
'when_worthwhile': 'Always if GPU available'
},
'MTCNN': {
'cpu_time': 45_ms,
'gpu_time': 18_ms,
'speedup': 2.5,
'when_worthwhile': 'High throughput scenarios'
},
'MediaPipe': {
'cpu_time': 12_ms,
'gpu_time': 8_ms,
'speedup': 1.5,
'when_worthwhile': 'Rarely - already fast on CPU'
}
}
# Cost-benefit analysis:
cpu_deployment = {
'instance': 't3.xlarge',
'hourly_cost': 0.16,
'throughput': 1.2, # FPS (RetinaFace)
'images_per_hour': 4_320
}
gpu_deployment = {
'instance': 'g4dn.xlarge',
'hourly_cost': 0.526,
'throughput': 15.4, # FPS (RetinaFace)
'images_per_hour': 55_440
}
# Efficiency comparison:
cpu_cost_per_1000_images = (0.16 / 4.32) # $0.037
gpu_cost_per_1000_images = (0.526 / 55.44) # $0.0095
# GPU is 3.9x cheaper per image despite higher instance cost
# Break-even calculation:
hourly_images_for_gpu_breakeven = 1000
# If processing >1000 images/hour, GPU is more economical
# Batch size optimization:
gpu_batch_optimization = {
'batch_size_1': 65_ms, # Single image
'batch_size_4': 88_ms, # 4 images (22ms each)
'batch_size_8': 140_ms, # 8 images (17.5ms each)
'batch_size_16': 245_ms, # 16 images (15.3ms each)
'optimal_batch': 8, # Balance throughput and latency
}5. Model Quantization for Mobile#
# Reduce model size and increase speed for mobile
quantization_impact = {
'float32_model': {
'precision': 32, # Bits per weight
'model_size': 10.5_MB,
'inference_time': 95_ms, # On mobile CPU
'accuracy': 0.94
},
'float16_model': {
'precision': 16, # Half precision
'model_size': 5.25_MB, # 50% smaller
'inference_time': 68_ms, # 1.4x faster
'accuracy': 0.94 # No accuracy loss
},
'int8_model': {
'precision': 8, # Integer quantization
'model_size': 2.6_MB, # 75% smaller
'inference_time': 28_ms, # 3.4x faster
'accuracy': 0.92 # 2% accuracy loss
}
}
# Mobile deployment comparison:
quantization_benefits = {
'app_size_reduction': '7.9 MB saved', # Significant for downloads
'battery_life_improvement': '2.4x', # Lower compute = less power
'inference_speed': '3.4x faster',
'trade_off': '2% accuracy loss' # Usually acceptable
}
# When quantization is essential:
quantization_required = [
'Mobile applications (app size matters)',
'Edge devices (limited compute)',
'Battery-powered (efficiency critical)',
'High-volume inference (cost reduction)'
]
# Quantization-aware training:
advanced_quantization = {
'method': 'Train model expecting quantization',
'accuracy_recovery': '~1%', # Recover most accuracy loss
'result': 'int8 with near float32 accuracy',
'effort': 'Requires retraining model'
}Benchmark Interpretation#
WIDER FACE Dataset Explained#
# Industry-standard face detection benchmark
wider_face_details = {
'total_images': 32_203,
'total_faces': 393_703,
'annotation': 'Bounding boxes for all faces',
'difficulty_levels': ['Easy', 'Medium', 'Hard'],
'evaluation_metric': 'Average Precision (AP)',
'why_important': 'Simulates real-world conditions'
}
# Difficulty characteristics:
difficulty_breakdown = {
'Easy': {
'face_size': 'Large (>100px)',
'occlusion': 'Minimal',
'pose': 'Frontal',
'lighting': 'Good',
'example_scenarios': [
'Portrait photos',
'ID photos',
'Close-up selfies',
'Professional headshots'
],
'percentage': '35% of dataset'
},
'Medium': {
'face_size': 'Medium (50-100px)',
'occlusion': 'Partial (sunglasses, partial profile)',
'pose': 'Slight angles',
'lighting': 'Variable',
'example_scenarios': [
'Group photos',
'Casual photos',
'Indoor events',
'Social media photos'
],
'percentage': '40% of dataset'
},
'Hard': {
'face_size': 'Small (<50px)',
'occlusion': 'Heavy (masks, extreme angles, poor lighting)',
'pose': 'Extreme angles',
'lighting': 'Poor',
'example_scenarios': [
'Surveillance footage',
'Crowd scenes',
'Distant cameras',
'Low-light conditions'
],
'percentage': '25% of dataset'
}
}
# Interpreting scores:
score_interpretation = {
'Easy > 0.95': 'Excellent - handles standard use cases',
'Medium > 0.90': 'Good - robust to typical variations',
'Hard > 0.80': 'Very good - handles difficult conditions',
'Hard < 0.70': 'Poor - will struggle in production'
}
# Algorithm benchmark comparison:
wider_face_results = {
'Haar Cascade': {
'easy': 0.85, 'medium': 0.60, 'hard': 0.30,
'interpretation': 'Good for easy cases, struggles with variations'
},
'Dlib HOG': {
'easy': 0.89, 'medium': 0.68, 'hard': 0.38,
'interpretation': 'Slightly better, still not robust'
},
'MTCNN': {
'easy': 0.95, 'medium': 0.88, 'hard': 0.72,
'interpretation': 'Robust across conditions'
},
'RetinaFace': {
'easy': 0.99, 'medium': 0.97, 'hard': 0.91,
'interpretation': 'Best-in-class across all conditions'
},
'MediaPipe': {
'easy': 0.96, 'medium': 0.89, 'hard': 0.73,
'interpretation': 'Excellent for mobile, good robustness'
}
}
# What score differences mean:
practical_impact = {
'0.85_vs_0.95_easy': {
'score_diff': 0.10,
'practical_impact': '10 missed faces per 100',
'when_matters': 'Photo albums - missing people'
},
'0.60_vs_0.88_medium': {
'score_diff': 0.28,
'practical_impact': '28 missed faces per 100',
'when_matters': 'Security - significant missed detections'
},
'0.30_vs_0.72_hard': {
'score_diff': 0.42,
'practical_impact': '42 missed faces per 100',
'when_matters': 'Surveillance - system unreliable'
}
}LFW (Labeled Faces in the Wild) Explained#
# Face recognition (not detection) benchmark
lfw_details = {
'purpose': 'Face recognition accuracy',
'total_images': 13_233,
'total_identities': 5_749,
'task': 'Same person or different person?',
'metric': 'Verification accuracy',
'why_relevant': 'Recognition follows detection'
}
# Recognition pipeline:
recognition_pipeline = {
'step_1_detection': 'Find faces in image',
'step_2_alignment': 'Align faces using landmarks',
'step_3_embedding': 'Generate identity vector',
'step_4_comparison': 'Compare vectors (same person?)',
'lfw_measures': 'Step 4 accuracy'
}
# Score interpretation:
lfw_accuracy_meaning = {
'> 99.5%': 'State-of-the-art, production-ready',
'99.0 - 99.5%': 'Excellent, suitable for most applications',
'97.0 - 99.0%': 'Good, acceptable for non-critical uses',
'< 97.0%': 'Poor, not suitable for production'
}
# Algorithm LFW scores:
lfw_results = {
'Dlib ResNet': {
'accuracy': 0.9938,
'interpretation': 'Excellent for face recognition',
'use_cases': 'Photo tagging, authentication'
},
'FaceNet': {
'accuracy': 0.9965,
'interpretation': 'State-of-the-art recognition',
'use_cases': 'Security, high-accuracy applications'
},
'ArcFace': {
'accuracy': 0.9982,
'interpretation': 'Best-in-class',
'use_cases': 'Critical applications, large-scale'
}
}
# Why 99%+ matters:
recognition_impact = {
'97%_accuracy': {
'false_accept_rate': 0.03, # 3% wrong identity
'practical_impact': '3 in 100 unlock attempts wrong person',
'security_level': 'Unacceptable for authentication'
},
'99.5%_accuracy': {
'false_accept_rate': 0.005, # 0.5% wrong identity
'practical_impact': '1 in 200 unlock attempts wrong person',
'security_level': 'Acceptable for consumer applications'
},
'99.8%_accuracy': {
'false_accept_rate': 0.002, # 0.2% wrong identity
'practical_impact': '1 in 500 unlock attempts wrong person',
'security_level': 'Suitable for high-security applications'
}
}
# Important distinction:
detection_vs_recognition_benchmarks = {
'WIDER_FACE': 'Detection - finding faces',
'LFW': 'Recognition - identifying faces',
'don_t_confuse': 'Good detection ≠ good recognition',
'example': 'MediaPipe: excellent detection, no recognition',
}Decision Framework Summary#
Quick Decision Tree#
def choose_face_detection_library(requirements):
"""
Systematic decision framework for face detection library selection
"""
# Real-time video applications
if requirements['application_type'] == 'real_time_video':
if requirements['fps_target'] >= 60:
return 'MediaPipe' # Only option for 60 FPS
elif requirements['fps_target'] >= 30:
if requirements['need_3d_mesh']:
return 'MediaPipe' # AR effects
elif requirements['accuracy_priority'] == 'high':
return 'MTCNN' # Best balance
else:
return 'OpenCV Haar' # Fastest
else: # FPS < 30
return 'Dlib HOG' # Good balance for lower FPS
# Batch photo processing
elif requirements['application_type'] == 'batch_processing':
if requirements['accuracy_priority'] == 'highest':
return 'RetinaFace' # Best accuracy
elif requirements['volume'] > 100_000:
return 'MTCNN' # Good accuracy, reasonable speed
else:
return 'Dlib HOG' # Fast enough for smaller batches
# Mobile applications
elif requirements['application_type'] == 'mobile':
if requirements['need_3d_mesh']:
return 'MediaPipe Face Mesh' # Only mobile 3D option
elif requirements['model_size_limit'] < 5:
return 'MediaPipe' # Small model
else:
return 'Dlib CNN (quantized)' # More accurate, larger
# Cloud/API decision
elif requirements['monthly_detections'] < 100_000:
return 'Face++ or Amazon Rekognition' # Cost-effective at low volume
# Embedded/edge devices
elif requirements['deployment'] == 'embedded':
if requirements['has_gpu']:
return 'MediaPipe' # Efficient
else:
return 'OpenCV Haar' # CPU-only lightweight option
# High-accuracy requirements
elif requirements['accuracy_priority'] == 'critical':
return 'RetinaFace + InsightFace' # Best accuracy
# Default fallback
else:
return 'MTCNN' # Good all-around choice
# Example usage:
requirements_video_conference = {
'application_type': 'real_time_video',
'fps_target': 30,
'need_3d_mesh': False,
'accuracy_priority': 'medium'
}
# Returns: 'MTCNN'
requirements_ar_filters = {
'application_type': 'mobile',
'need_3d_mesh': True,
'model_size_limit': 10
}
# Returns: 'MediaPipe Face Mesh'
requirements_photo_album = {
'application_type': 'batch_processing',
'accuracy_priority': 'highest',
'volume': 50_000
}
# Returns: 'RetinaFace'Use Case Matrix#
# Comprehensive use case to library mapping
use_case_recommendations = {
'Security Monitoring': {
'recommended': 'MTCNN',
'rationale': 'High recall critical, real-time capable',
'alternatives': ['RetinaFace (if can afford latency)'],
'avoid': 'Haar Cascade (too many missed detections)'
},
'Photo Album Organization': {
'recommended': 'RetinaFace',
'rationale': 'Highest accuracy, batch processing acceptable',
'alternatives': ['MTCNN (faster, slightly less accurate)'],
'avoid': 'Haar Cascade (miss too many faces)'
},
'Video Conferencing': {
'recommended': 'MediaPipe',
'rationale': 'Real-time, efficient, precise segmentation',
'alternatives': ['Dlib HOG (simpler, less accurate)'],
'avoid': 'RetinaFace (too slow for real-time)'
},
'AR Filters (Snapchat/Instagram-style)': {
'recommended': 'MediaPipe Face Mesh',
'rationale': 'Only option for 3D mesh on mobile',
'alternatives': ['None - unique capability'],
'avoid': 'All 2D-only detectors'
},
'Attendance System': {
'recommended': 'MTCNN + ArcFace',
'rationale': 'High accuracy detection + recognition',
'alternatives': ['InsightFace (all-in-one)'],
'avoid': 'Haar Cascade (too many false negatives)'
},
'Mobile Photo App': {
'recommended': 'MediaPipe',
'rationale': 'Small model, battery-efficient',
'alternatives': ['Dlib CNN quantized (more accurate)'],
'avoid': 'RetinaFace (too large for mobile)'
},
'Embedded Security Camera': {
'recommended': 'OpenCV Haar',
'rationale': 'Lightweight, no GPU required',
'alternatives': ['Dlib HOG (better accuracy)'],
'avoid': 'Deep learning models (need GPU)'
},
'MVP/Prototype': {
'recommended': 'Face++ API',
'rationale': 'Zero infrastructure, fast integration',
'alternatives': ['Amazon Rekognition', 'Azure Face'],
'avoid': 'Self-hosting (premature optimization)'
},
'High-Volume Cloud Service': {
'recommended': 'RetinaFace (self-hosted GPU)',
'rationale': 'Best accuracy, economical at scale',
'alternatives': ['MTCNN (faster, less accurate)'],
'avoid': 'Cloud APIs (expensive at scale)'
},
'Driver Monitoring': {
'recommended': 'MediaPipe Face Mesh',
'rationale': 'Precise head pose, drowsiness detection',
'alternatives': ['Dlib 68-point (simpler)'],
'avoid': '5-point detection (insufficient detail)'
}
}Performance vs Accuracy Matrix#
# Visual decision matrix
library_positioning = {
'OpenCV Haar': {
'speed': 'Fastest',
'accuracy': 'Lowest',
'when_choose': 'Speed critical, conditions controlled'
},
'Dlib HOG': {
'speed': 'Very Fast',
'accuracy': 'Medium',
'when_choose': 'Balance of speed and accuracy'
},
'MediaPipe': {
'speed': 'Fast',
'accuracy': 'High',
'when_choose': 'Mobile or real-time with good accuracy'
},
'MTCNN': {
'speed': 'Medium',
'accuracy': 'High',
'when_choose': 'Real-time with high accuracy'
},
'RetinaFace': {
'speed': 'Slow',
'accuracy': 'Highest',
'when_choose': 'Batch processing, accuracy critical'
}
}
# Cost-benefit analysis
cost_benefit_matrix = {
'Lowest Cost': {
'options': ['OpenCV Haar', 'Dlib HOG'],
'deployment': 'CPU-only, low compute',
'trade_off': 'Lower accuracy'
},
'Best Value': {
'options': ['MTCNN', 'MediaPipe'],
'deployment': 'CPU or light GPU',
'trade_off': 'Balanced performance'
},
'Highest Performance': {
'options': ['RetinaFace', 'InsightFace'],
'deployment': 'GPU required',
'trade_off': 'Higher infrastructure cost'
}
}Conclusion#
Face detection library selection is a strategic system design decision affecting:
- Direct user experience impact: Algorithm latency determines application responsiveness
- Accuracy-driven outcomes: Detection miss rate affects system reliability and user trust
- Deployment feasibility: Model size and compute requirements determine platform compatibility
- Economic efficiency: Wrong algorithm choice can cost 5-10x more in infrastructure or lost users
- Feature capabilities: 2D vs 3D, landmark detail, recognition integration
Understanding face detection fundamentals helps contextualize why algorithm and library selection creates measurable business value through improved user experience, system reliability, and operational efficiency, making it a high-ROI architectural decision.
Key Insight: Face detection is performance-accuracy-cost optimization problem - the “best” library depends entirely on your specific constraints (real-time, batch, mobile, accuracy, cost). There is no universal best choice, only the best choice for your use case.
Date compiled: November 21, 2025
S1: Rapid Discovery
Face Detection & Recognition Libraries: S1 Rapid Discovery#
Research Overview#
This directory contains comprehensive S1 Rapid Discovery research on face detection and recognition libraries for experiment 1.091.2 in the spawn-solutions research framework.
Research Date: January 2025 Research Type: Generic reference material (Hardware Store for Software) Scope: Comparative analysis of 8 face detection/recognition solutions
Documents in This Research#
Individual Library Analyses#
mediapipe.md (364 lines)
- Google’s MediaPipe Face Detection & Mesh
- 468-point 3D face mesh, mobile-optimized
- Best for: AR filters, mobile apps, 3D face tracking
dlib.md (468 lines)
- Dlib Face Detection & Recognition
- 68-point landmarks, 99.38% LFW recognition accuracy
- Best for: Face recognition, landmark detection, desktop apps
insightface.md (527 lines)
- InsightFace 2D & 3D Face Analysis
- State-of-the-art recognition (99.83% LFW), ArcFace method
- Best for: Production face recognition, high accuracy requirements
mtcnn.md (485 lines)
- Multi-task Cascaded Convolutional Networks
- Lightweight (2 MB), cascade detector
- Best for: Legacy systems, embedded devices, educational
retinaface.md (515 lines)
- RetinaFace Single-stage Dense Face Localisation
- Highest detection accuracy (91.4% WIDER FACE hard)
- Best for: Challenging conditions, occlusions, production systems
opencv.md (562 lines)
- OpenCV Face Detection Methods (Haar, LBP, DNN)
- Traditional (fast CPU) and modern (DNN) approaches
- Best for: Quick prototyping, embedded systems, universal compatibility
commercial-apis.md (747 lines)
- Face++ API (Megvii) and Amazon Rekognition
- Cloud-based, comprehensive face attributes
- Best for: MVPs, no infrastructure, need age/gender/emotion
Synthesis & Decision Framework#
- synthesis.md (609 lines)
- Master comparison table across all libraries
- Decision framework: “Choose X if you need Y”
- Accuracy vs speed spectrum with benchmarks
- Use case patterns: Security, photo organization, AR, attendance, etc.
- Self-hosted vs cloud trade-offs
- Quick decision tree for choosing libraries
- Performance optimization tips
- Privacy implications (GDPR-compliant options)
Quick Reference#
By Primary Need#
| Need | Recommended Library | Document |
|---|---|---|
| Highest detection accuracy | RetinaFace (91.4%) | retinaface.md |
| Highest recognition accuracy | InsightFace (99.83% LFW) | insightface.md |
| Dense 3D face mesh | MediaPipe (468 points) | mediapipe.md |
| 68-point landmarks | Dlib | dlib.md |
| Fastest CPU detection | OpenCV Haar Cascades | opencv.md |
| Mobile/web support | MediaPipe | mediapipe.md |
| Face attributes (age/gender) | Face++, AWS Rekognition | commercial-apis.md |
Smallest model (<2 MB) | MTCNN, RetinaFace (MobileNet) | mtcnn.md, retinaface.md |
| Privacy-friendly (on-device) | All self-hosted libraries | See individual docs |
| Quick MVP (cloud) | AWS Rekognition, Face++ | commercial-apis.md |
Accuracy Benchmarks#
| Library | Detection Accuracy | Recognition Accuracy (LFW) |
|---|---|---|
| RetinaFace | 91.4% (WIDER FACE hard) | N/A (detection only) |
| InsightFace | 91.4% (uses RetinaFace) | 99.83% |
| MediaPipe | 99.3% (comparative study) | N/A (detection only) |
| Dlib | Excellent (CNN), Good (HOG) | 99.38% |
| MTCNN | 97.56% AUC (2016) | N/A (detection only) |
| OpenCV DNN | 85-95% | N/A (weak built-in) |
| OpenCV Haar | 70-85% (frontal only) | N/A (weak built-in) |
| Face++ | 99%+ (proprietary) | 99%+ (proprietary) |
| AWS Rekognition | High (production-grade) | High (production-grade) |
Speed Comparison (CPU)#
| Library | Speed (FPS) | Notes |
|---|---|---|
| OpenCV Haar | 30+ FPS | Fastest, frontal faces only |
| Dlib HOG | 30+ FPS | Fast, frontal faces only |
| MediaPipe | 60-100 FPS | Optimized, but 468 landmarks |
| OpenCV DNN | 15-30 FPS | Good balance |
| MTCNN | 5-15 FPS | Cascade design |
| RetinaFace | 3-20 FPS | Depends on backbone |
| Dlib CNN | 1-3 FPS | Requires GPU for real-time |
Research Methodology#
Information Sources#
- Official documentation and GitHub repositories
- Academic papers (CVPR, ECCV, ICCV)
- Benchmark datasets: WIDER FACE, LFW, IJB-B/C, 300W, AFLW
- Community implementations and performance reports
- Web search (2024-2025 current information)
Evaluation Criteria#
- Detection accuracy (WIDER FACE benchmark)
- Recognition accuracy (LFW benchmark)
- Landmark quality (point count, 2D vs 3D)
- Speed (FPS on CPU and GPU)
- Model size (MB)
- Platform support (desktop, mobile, web, edge)
- API usability and documentation
- Licensing and cost
- Privacy implications
- Production readiness
How to Use This Research#
For Decision Making#
- Start with: synthesis.md - Decision framework and comparison table
- Use case matching: Find your scenario in the “Generic Use Case Patterns” section
- Deep dive: Read individual library documents for implementation details
For Implementation#
- Choose library based on synthesis recommendations
- Review code examples in individual library documents
- Check platform support for your target deployment
- Verify licensing for commercial use
For Benchmarking#
- Compare metrics in synthesis master table
- Review accuracy benchmarks (WIDER FACE, LFW)
- Check speed comparisons for your hardware profile
Key Insights#
Top 3 for Each Category#
Detection Accuracy:
- RetinaFace (ResNet-152): 91.4% WIDER FACE hard
- InsightFace (RetinaFace): 91.4% WIDER FACE hard
- MediaPipe: 99.3% comparative study
Recognition Accuracy:
- InsightFace (ArcFace): 99.83% LFW
- Dlib: 99.38% LFW
- Face++: 99%+ (proprietary)
Mobile Performance:
- MediaPipe: 30-60 FPS,
<10MB, official SDKs - RetinaFace (MobileNet): 1.7 MB, 60+ FPS GPU
- MTCNN: 2 MB, acceptable mobile performance
Privacy-Friendly (On-device):
- MediaPipe: No telemetry, Apache 2.0
- Dlib: No telemetry, Boost License
- InsightFace: ONNX Runtime, self-hosted
Cost-Effective (Self-hosted):
- OpenCV: Free, Apache 2.0, minimal dependencies
- MediaPipe: Free, Apache 2.0,
<10MB - MTCNN: Free, MIT License, 2 MB
Document Statistics#
- Total documents: 8 (7 libraries + 1 synthesis)
- Total lines: 4,277
- Total size: 148 KB
- Code examples: 30+ Python examples across all documents
- Benchmarks cited: WIDER FACE, LFW, IJB-B/C, 300W, AFLW
- Libraries covered: 8 (6 self-hosted + 2 commercial APIs)
Generic Use Case Examples#
These are generic patterns applicable to any developer, NOT client-specific:
- Security Systems: Surveillance, access control, attendance tracking
- Photo Organization: Face clustering, search by person, album tagging
- AR Applications: Filters, effects, virtual try-on, face tracking
- Video Conferencing: Background blur, beautification, face position
- Retail Analytics: Customer demographics, emotion analysis
- Age Verification: Online services, retail compliance
- Social Media: Face tagging, verification, content moderation
Navigation Tips#
- New to face detection? Start with opencv.md (easiest) or mediapipe.md (modern)
- Need highest accuracy? Read retinaface.md (detection) and insightface.md (recognition)
- Building MVP quickly? Check commercial-apis.md for cloud options
- Privacy concerns? All self-hosted libraries support on-device processing (see synthesis)
- Overwhelmed by choices? Use the decision tree in synthesis.md
Updates & Maintenance#
This research reflects the state of face detection/recognition libraries as of January 2025. Key libraries are actively maintained:
- MediaPipe: Google actively developing (2024-2025)
- Dlib: Stable, mature (10+ years)
- InsightFace: Actively maintained (2024-2025)
- RetinaFace: Community implementations maintained
- OpenCV: Very actively maintained (2024-2025)
- MTCNN: Stable, less active (surpassed by newer methods)
- Face++, AWS Rekognition: Commercial services, regularly updated
Contact & Feedback#
This research is part of the spawn-solutions research framework, experiment 1.091.2.
For questions or additions, consult the individual library documentation and GitHub repositories linked in each document.
Research completed: January 2025 Framework: spawn-solutions Experiment: 1.091.2-face-detection Phase: S1 Rapid Discovery
S1: Rapid Discovery — Approach#
S1 research date: January 2025, per this directory’s README. This file written: 2026-08-25, alongside S2-S4.
Note on This File#
S1 was completed without an approach.md. This file records what S1
actually did, reconstructed from its README, its per-library files and its
synthesis, so that a reader can weigh its findings against its method. It
does not restate S1’s conclusions — those are in synthesis.md — and it
does not revise them. The revisions are in
../S2-comprehensive/recommendation.md.
What S1 Set Out to Answer#
S1’s stated scope, from its README, is a “Comparative analysis of 8 face detection/recognition solutions”. The eight are MediaPipe, dlib, InsightFace, MTCNN, RetinaFace, OpenCV, Face++ and Amazon Rekognition — six self-hosted and two hosted.
Its evaluation criteria, as listed in the README:
- Detection accuracy (WIDER FACE benchmark)
- Recognition accuracy (LFW benchmark)
- Landmark quality (point count, 2D vs 3D)
- Speed (FPS on CPU and GPU)
- Model size (MB)
- Platform support (desktop, mobile, web, edge)
- API usability and documentation
- Licensing and cost
- Privacy implications
- Production readiness
S1’s Sources, as It Describes Them#
The README lists:
- Official documentation and GitHub repositories
- Academic papers (CVPR, ECCV, ICCV)
- Benchmark datasets: WIDER FACE, LFW, IJB-B/C, 300W, AFLW
- Community implementations and performance reports
- Web search (2024-2025 current information)
The provenance of individual figures is not recorded. S1’s tables carry
numbers without per-figure attribution, and the S2 pass was able to trace
some of them to a primary source and not others. Which is which is listed in
../S2-comprehensive/recommendation.md under “Claims with no source”.
What S1 Produced#
| File | Lines | Content |
|---|---|---|
README.md | 245 | Index, quick-reference tables, methodology note |
mediapipe.md | 364 | Google MediaPipe Face Detection and Face Mesh |
dlib.md | 468 | dlib detection, landmarks and recognition |
insightface.md | 527 | InsightFace, ArcFace, SCRFD, the buffalo packs |
mtcnn.md | 485 | MTCNN cascade detector |
retinaface.md | 515 | RetinaFace |
opencv.md | 562 | Haar, LBP and DNN methods |
commercial-apis.md | 747 | Face++ and Amazon Rekognition |
synthesis.md | 609 | Master comparison, decision framework, decision tree |
Roughly 4,500 lines, well above the 100-200 lines per library that ADDING-RESEARCH sets for S1, and containing extensive code examples that belong in S2 under the current methodology.
Where S1’s Structure Shaped Its Conclusions#
Two structural choices in S1 propagate into its findings and are worth naming here, because they explain the corrections rather than merely listing them.
One comparison table for three jobs. S1’s master table has a single “Detection Accuracy” column and a single “Recognition” column, populated with figures from WIDER FACE and LFW. Those benchmarks measure different operations and their numbers are not commensurable. The table’s shape invites the reader to compare across them.
One license row per project. S1’s licensing summary gives each project one license — “Dlib: Boost”, “RetinaFace: MIT (implementations)”, “InsightFace: Mixed”. In this category the package license, the weights license and the training-data license are granted separately by different parties, and for three of the projects in S1’s table they differ. S2 and S4 resolve each chain against its own source.
Coverage Gaps#
S1’s eight options omit four things that a 2026 reader will encounter:
ageitgey/face_recognition— 56,682 stars, the most-starred project in the category, no release since 2020-02-20.- DeepFace — 23,337 stars, actively maintained, 487,770 downloads in the 30 days to 2026-08-25.
- YuNet (
cv2.FaceDetectorYN) — mentioned twice in one-line asides, never analyzed. MIT weights, 227 KB. - SFace (
cv2.FaceRecognizerSF) — absent entirely. Apache-2.0 weights, in the core OpenCV library since 4.5.4.
The last two are the material omission, because together they are the only detection-plus-recognition pair in the category with permissive licenses on the weights.
How to Read S1 Now#
S1’s per-library files remain useful as descriptions of what each project
is and how its API is shaped. Its tables, its accuracy figures and its
licensing summary have been superseded. The corrections are collected in
../S2-comprehensive/recommendation.md, and the per-project detail is in
the corresponding S2 file.
Commercial Face Detection & Recognition APIs#
Overview#
This document compares two leading commercial face detection and recognition APIs: Face++ (Megvii) and Amazon Rekognition (AWS). These cloud-based services offer comprehensive face analysis capabilities without requiring self-hosted infrastructure.
Face++ API (Megvii)#
1. Overview#
What it is: Face++ is a leading AI computer vision platform from Megvii (Chinese AI company), providing cloud-based face detection, recognition, and analysis APIs. Known for high accuracy and comprehensive feature set.
Maintainer: Megvii Technology (Face++ team)
License: Commercial (proprietary)
Primary Language: API-based (language agnostic), SDKs for Python, Java, iOS, Android, JavaScript
Active Development Status:
- Website: https://www.faceplusplus.com
- Status: Production-ready, widely deployed (especially in Asia)
- Used by: Alibaba, Lenovo, and thousands of developers
2. Core Capabilities#
Face Detection#
- High accuracy: 99%+ detection rate
- Multi-face detection (up to 100 faces per image)
- Bounding box with confidence scores
- Robust to varied poses, lighting, occlusions
Facial Landmarks#
- 83-point landmarks: Dense facial feature points
- 106-point landmarks: Even more detailed (premium)
- Eyes, eyebrows, nose, mouth, face contour
Face Recognition/Identification#
- 1:1 verification: Compare two faces (same person or not)
- 1:N identification: Search face in database
- Face clustering and grouping
- High accuracy (99%+ in controlled conditions)
Face Attributes#
- Age estimation: Predicted age
- Gender classification: Male/female
- Emotion detection: Happy, sad, angry, surprised, disgusted, calm, confused (7 emotions)
- Face quality: Blur, occlusion, lighting assessment
- Facial features: Glasses, beard, mask detection
- Beauty score: Aesthetic rating
- Head pose: Yaw, pitch, roll angles
- Eye status: Open/closed
- Mouth status: Open/closed
- Ethnicity: Racial classification (available in some regions)
3D Face Reconstruction#
- 3D face modeling: Available in advanced tiers
- 3D pose estimation
- Dense 3D mesh generation
Real-time Performance#
- Cloud API: Depends on network latency
- Typical response: 200-500 ms per API call
- On-premise SDK: Available for low-latency requirements
3. Technical Architecture#
Underlying Models#
- Proprietary deep learning models
- Trained on millions of faces
- Multi-task learning for detection + attributes
- Regular model updates (no user intervention needed)
API Endpoints#
- Face Detection:
/detect- Detect faces and attributes - Face Comparison:
/compare- Compare two faces (1:1) - Face Search:
/search- Find face in faceset (1:N) - Faceset Management: Create, add, remove faces from database
- Face Landmarks: Dense landmark extraction
Platform Support#
- Cloud API: Accessible from anywhere
- SDK support:
- iOS (Objective-C, Swift)
- Android (Java, Kotlin)
- Python
- Java
- JavaScript
- C++
- On-premise: Enterprise deployment available
Model Size / Deployment#
- Cloud-based: No local models
- On-premise SDK: Model sizes not publicly disclosed
Dependencies#
- Cloud API: HTTP client only (curl, requests, etc.)
- SDKs: Language-specific dependencies
4. Performance Benchmarks#
Accuracy#
- Face detection: 99%+ accuracy
- Face recognition: Industry-leading (exact benchmarks proprietary)
- Low false positive rate: Optimized for production
- Robust to: Lighting, angles, occlusions, age variations
Speed#
- API latency: 200-500 ms (depends on network, server location)
- Batch processing: Available for large volumes
- On-premise: Sub-50 ms with local deployment
Resource Requirements#
- Client-side: Minimal (API calls only)
- Server-side: Managed by Megvii (scalable)
5. API & Usability#
Python Example: Face Detection#
import requests
import json
# API credentials
API_KEY = 'your_api_key'
API_SECRET = 'your_api_secret'
# Face++ API endpoint
detect_url = 'https://api-us.faceplusplus.com/facepp/v3/detect'
# Image file or URL
image_path = 'photo.jpg'
# Parameters
params = {
'api_key': API_KEY,
'api_secret': API_SECRET,
'return_landmark': 1,
'return_attributes': 'gender,age,emotion,beauty,facequality'
}
# Upload image
files = {'image_file': open(image_path, 'rb')}
# Make API call
response = requests.post(detect_url, data=params, files=files)
result = response.json()
# Parse results
if 'faces' in result:
for face in result['faces']:
# Bounding box
bbox = face['face_rectangle']
print(f"Face at: ({bbox['left']}, {bbox['top']}), "
f"size: {bbox['width']}x{bbox['height']}")
# Attributes
attrs = face['attributes']
print(f" Age: {attrs['age']['value']}")
print(f" Gender: {attrs['gender']['value']}")
print(f" Emotion: {max(attrs['emotion'], key=attrs['emotion'].get)}")
print(f" Beauty: {attrs['beauty']['female_score']}/{attrs['beauty']['male_score']}")
# Landmarks
landmarks = face['landmark']
print(f" Landmarks: {len(landmarks)} points")Python Example: Face Comparison (1:1)#
import requests
API_KEY = 'your_api_key'
API_SECRET = 'your_api_secret'
compare_url = 'https://api-us.faceplusplus.com/facepp/v3/compare'
# Compare two images
params = {
'api_key': API_KEY,
'api_secret': API_SECRET
}
files = {
'image_file1': open('person1.jpg', 'rb'),
'image_file2': open('person2.jpg', 'rb')
}
response = requests.post(compare_url, data=params, files=files)
result = response.json()
# Parse similarity
confidence = result['confidence']
threshold = result['thresholds']['1e-5'] # Recommended threshold
if confidence > threshold:
print(f"Same person! Confidence: {confidence:.2f}")
else:
print(f"Different people. Confidence: {confidence:.2f}")Python Example: Face Search (1:N)#
import requests
API_KEY = 'your_api_key'
API_SECRET = 'your_api_secret'
# 1. Create a faceset
create_faceset_url = 'https://api-us.faceplusplus.com/facepp/v3/faceset/create'
faceset_params = {
'api_key': API_KEY,
'api_secret': API_SECRET,
'display_name': 'Employee Database'
}
response = requests.post(create_faceset_url, data=faceset_params)
faceset_token = response.json()['faceset_token']
# 2. Add faces to faceset (from known people)
add_face_url = 'https://api-us.faceplusplus.com/facepp/v3/faceset/addface'
for person_image in ['alice.jpg', 'bob.jpg', 'charlie.jpg']:
# First detect the face
detect_response = requests.post(detect_url, data={
'api_key': API_KEY,
'api_secret': API_SECRET
}, files={'image_file': open(person_image, 'rb')})
face_token = detect_response.json()['faces'][0]['face_token']
# Add to faceset
requests.post(add_face_url, data={
'api_key': API_KEY,
'api_secret': API_SECRET,
'faceset_token': faceset_token,
'face_tokens': face_token
})
# 3. Search for a face in the faceset
search_url = 'https://api-us.faceplusplus.com/facepp/v3/search'
search_params = {
'api_key': API_KEY,
'api_secret': API_SECRET,
'faceset_token': faceset_token
}
search_files = {'image_file': open('query.jpg', 'rb')}
response = requests.post(search_url, data=search_params, files=search_files)
result = response.json()
# Parse results
if result['results']:
best_match = result['results'][0]
confidence = best_match['confidence']
print(f"Match found with confidence: {confidence:.2f}")
else:
print("No match found")Learning Curve#
Rating: Beginner-friendly (2/5 difficulty)
- Simple REST API
- Good documentation
- SDKs for common languages
- Dashboard for API key management
Documentation Quality#
Rating: 8/10
- Comprehensive API documentation
- Code examples in multiple languages
- Interactive console for testing
- Good community support
- Some documentation in Chinese (English available)
6. Pricing Model#
Pay-per-call#
- Free tier: Available (limited calls/month)
- API pricing: Starting at $100/day (tiered pricing)
- Volume discounts: Available for large-scale users
- No upfront costs: Pay as you go
Faceset Storage#
- Face database storage: May incur additional charges
- Free tier includes limited storage
On-premise Licensing#
- Enterprise licensing available
- Contact sales for pricing
7. Use Case Fit#
Best For#
- Cloud-first applications: No infrastructure management
- Comprehensive face attributes: Age, gender, emotion, beauty
- Asia-Pacific deployments: Strong server presence in Asia
- Quick prototyping: No setup, immediate API access
- Face verification: KYC, authentication
- Face search: Find person in database
- Emotion analysis: Customer sentiment, user engagement
Limitations#
- Network dependency: Requires internet connection
- Privacy concerns: Data sent to Megvii servers
- Latency: 200-500 ms API calls (not sub-10ms)
- Cost at scale: High-volume can be expensive
- Vendor lock-in: Proprietary API
- Regional compliance: Data residency concerns (China-based company)
Amazon Rekognition (AWS)#
1. Overview#
What it is: Amazon Rekognition is a fully managed computer vision service from AWS, providing face detection, analysis, recognition, and comparison via cloud API. Part of the AWS ecosystem.
Maintainer: Amazon Web Services (AWS)
License: Commercial (proprietary)
Primary Language: API-based (language agnostic), AWS SDKs for Python (boto3), Java, JavaScript, .NET, PHP, Ruby, Go
Active Development Status:
- Website: https://aws.amazon.com/rekognition/
- Status: Production-ready, enterprise-grade
- Used by: Thousands of AWS customers globally
2. Core Capabilities#
Face Detection#
- Accurate face detection in images and videos
- Bounding boxes with confidence scores
- Multi-face detection
- Robust to varied conditions
Facial Landmarks#
- Eyes, eyebrows, nose, mouth
- Face contour points
- Less detailed than Face++ (fewer points)
Face Recognition/Identification#
- Face comparison: Compare two faces (1:1)
- Face search: Search in face collection (1:N)
- Face indexing: Create searchable face database
- Real-time face recognition in video streams
Face Attributes#
- Gender: Male/female classification
- Age range: Estimated age bracket
- Emotions: Happy, sad, angry, surprised, disgusted, calm, confused (7 emotions)
- Eye status: Open/closed, eyeglasses, sunglasses
- Facial hair: Beard, mustache
- Face quality: Brightness, sharpness
- Head pose: Pitch, roll, yaw
- Mouth status: Open/closed, smile
- Face occlusion: Detected occlusions
3D Face Reconstruction#
- Not provided
Real-time Performance#
- API latency: 100-500 ms (depends on region, network)
- Video analysis: Near real-time streaming support
- Batch processing: Supported for images and videos
3. Technical Architecture#
Underlying Models#
- Proprietary AWS deep learning models
- Trained on millions of diverse images
- Regular updates (automatic, no user action)
- Multi-task learning architecture
API Operations#
- DetectFaces: Detect faces and attributes
- CompareFaces: Compare two faces (1:1)
- SearchFacesByImage: Find face in collection (1:N)
- IndexFaces: Add face to collection
- CreateCollection: Create face database
- RecognizeCelebrities: Identify famous people
- Video analysis: DetectFaces, SearchFaces in video
Platform Support#
- Cloud API: AWS global infrastructure
- AWS SDKs:
- Python (boto3)
- Java
- JavaScript (Node.js, browser)
- .NET
- PHP, Ruby, Go
- AWS Lambda: Serverless integration
- AWS ecosystem: S3, CloudWatch, SNS integration
Model Size / Deployment#
- Cloud-only: No local models
- Edge deployment: AWS Panorama (specialized hardware)
Dependencies#
- AWS SDK: boto3 (Python), aws-sdk (JavaScript), etc.
- AWS credentials: IAM access keys
4. Performance Benchmarks#
Accuracy#
- Face detection: High accuracy across diverse conditions
- Face recognition: Robust, production-grade
- Attribute detection: Improved accuracy (2024 updates)
- Celebrity recognition: 10,000+ celebrities
- Accuracy improvements: Ongoing (recent enhancements to gender, emotion detection)
Speed#
- API latency: 100-500 ms (region-dependent)
- Video processing: Near real-time
- Batch: Efficient for large volumes
Resource Requirements#
- Client-side: Minimal (API calls only)
- Server-side: Fully managed by AWS (auto-scaling)
5. API & Usability#
Python Example: Face Detection#
import boto3
import json
# Initialize Rekognition client
rekognition = boto3.client('rekognition', region_name='us-east-1')
# Detect faces in image
with open('photo.jpg', 'rb') as image_file:
image_bytes = image_file.read()
response = rekognition.detect_faces(
Image={'Bytes': image_bytes},
Attributes=['ALL'] # Include all face attributes
)
# Parse results
for face in response['FaceDetails']:
# Bounding box
bbox = face['BoundingBox']
print(f"Face at: ({bbox['Left']:.2f}, {bbox['Top']:.2f}), "
f"size: {bbox['Width']:.2f}x{bbox['Height']:.2f}")
# Confidence
print(f" Confidence: {face['Confidence']:.2f}%")
# Age range
age_range = face['AgeRange']
print(f" Age: {age_range['Low']}-{age_range['High']}")
# Gender
gender = face['Gender']['Value']
gender_conf = face['Gender']['Confidence']
print(f" Gender: {gender} ({gender_conf:.2f}%)")
# Emotions
emotions = face['Emotions']
top_emotion = max(emotions, key=lambda x: x['Confidence'])
print(f" Emotion: {top_emotion['Type']} ({top_emotion['Confidence']:.2f}%)")
# Facial features
print(f" Beard: {face['Beard']['Value']}")
print(f" Eyeglasses: {face['Eyeglasses']['Value']}")
print(f" Smile: {face['Smile']['Value']}")Python Example: Face Comparison (1:1)#
import boto3
rekognition = boto3.client('rekognition', region_name='us-east-1')
# Compare two faces
with open('person1.jpg', 'rb') as source_image:
source_bytes = source_image.read()
with open('person2.jpg', 'rb') as target_image:
target_bytes = target_image.read()
response = rekognition.compare_faces(
SourceImage={'Bytes': source_bytes},
TargetImage={'Bytes': target_bytes},
SimilarityThreshold=80 # Minimum similarity to return match
)
# Parse results
if response['FaceMatches']:
for match in response['FaceMatches']:
similarity = match['Similarity']
print(f"Match found! Similarity: {similarity:.2f}%")
else:
print("No match found (below threshold)")
# Unmatched faces
if response['UnmatchedFaces']:
print(f"{len(response['UnmatchedFaces'])} unmatched faces in target image")Python Example: Face Search (1:N)#
import boto3
rekognition = boto3.client('rekognition', region_name='us-east-1')
# 1. Create a collection
collection_id = 'employee-collection'
rekognition.create_collection(CollectionId=collection_id)
# 2. Index faces from known people
for person_name, image_path in [('Alice', 'alice.jpg'), ('Bob', 'bob.jpg')]:
with open(image_path, 'rb') as image_file:
image_bytes = image_file.read()
response = rekognition.index_faces(
CollectionId=collection_id,
Image={'Bytes': image_bytes},
ExternalImageId=person_name, # Person's name
DetectionAttributes=['ALL']
)
print(f"Indexed {person_name}: {response['FaceRecords'][0]['Face']['FaceId']}")
# 3. Search for a face in the collection
with open('query.jpg', 'rb') as image_file:
image_bytes = image_file.read()
response = rekognition.search_faces_by_image(
CollectionId=collection_id,
Image={'Bytes': image_bytes},
MaxFaces=5,
FaceMatchThreshold=80 # Minimum similarity
)
# Parse results
if response['FaceMatches']:
for match in response['FaceMatches']:
similarity = match['Similarity']
face_id = match['Face']['FaceId']
external_id = match['Face']['ExternalImageId']
print(f"Match: {external_id}, Similarity: {similarity:.2f}%")
else:
print("No match found")Python Example: Video Face Detection#
import boto3
import time
rekognition = boto3.client('rekognition', region_name='us-east-1')
s3 = boto3.client('s3')
# 1. Upload video to S3
bucket_name = 'my-video-bucket'
video_key = 'video.mp4'
s3.upload_file('video.mp4', bucket_name, video_key)
# 2. Start face detection job
response = rekognition.start_face_detection(
Video={'S3Object': {'Bucket': bucket_name, 'Name': video_key}},
NotificationChannel={
'SNSTopicArn': 'arn:aws:sns:us-east-1:123456789:RekognitionTopic',
'RoleArn': 'arn:aws:iam::123456789:role/RekognitionRole'
}
)
job_id = response['JobId']
print(f"Started job: {job_id}")
# 3. Wait for job completion
while True:
response = rekognition.get_face_detection(JobId=job_id)
status = response['JobStatus']
if status in ['SUCCEEDED', 'FAILED']:
break
time.sleep(5)
# 4. Get results
if status == 'SUCCEEDED':
for face in response['Faces']:
timestamp = face['Timestamp']
face_detail = face['Face']
bbox = face_detail['BoundingBox']
print(f"Face at {timestamp}ms: confidence {face_detail['Confidence']:.2f}%")Learning Curve#
Rating: Intermediate (3/5 difficulty)
- Requires AWS account and IAM setup
- boto3 (Python SDK) straightforward
- Good documentation
- AWS ecosystem knowledge helpful
Documentation Quality#
Rating: 9/10
- Excellent AWS documentation
- Code examples in all SDK languages
- Tutorials and workshops
- Active AWS forums
- Links:
- Docs: https://docs.aws.amazon.com/rekognition/
- Getting started: https://aws.amazon.com/rekognition/getting-started/
6. Pricing Model#
Pay-per-use (Images)#
- Free tier: 5,000 images/month (first 12 months)
- First 1 million images/month: $1.00 per 1,000 images
- Next 9 million: $0.80 per 1,000 images
- Next 90 million: $0.60 per 1,000 images
- Over 100 million: $0.40 per 1,000 images
Video Analysis#
- Separate pricing for video processing
- Per-minute charges
Face Collection Storage#
- First 1,000 faces/month: Free
- Additional faces: $0.01 per 1,000 faces stored per month
Free Tier (New Customers, July 2025+)#
- $200 AWS Free Tier credits applicable to Rekognition
Cost Example#
- 10,000 images/month: $10/month
- 100,000 images/month: $92/month
- 1 million images/month: $1,000/month
7. Use Case Fit#
Best For#
- AWS ecosystem: Already using AWS services
- Enterprise applications: Compliance, scalability, reliability
- Global deployments: AWS regions worldwide
- Video analysis: Real-time face detection in streams
- Serverless: AWS Lambda integration
- KYC/verification: Banking, fintech
- Content moderation: User-generated content
- Access control: Security systems
- Celebrity recognition: Media, entertainment
Limitations#
- Network dependency: Requires internet
- Privacy concerns: Data sent to AWS (can use encryption)
- Latency: 100-500 ms (not real-time on-device)
- Cost at scale: Can be expensive for high volumes
- AWS-specific: Vendor lock-in to AWS ecosystem
- No 3D reconstruction: Only 2D analysis
- Limited landmarks: Fewer points than Face++ or MediaPipe
Face++ vs Amazon Rekognition: Comparison#
Feature Comparison Table#
| Feature | Face++ | Amazon Rekognition |
|---|---|---|
| Detection Accuracy | 99%+ | High (AWS-grade) |
| Facial Landmarks | 83-106 points | Basic points |
| Face Recognition (1:1) | ✓ | ✓ |
| Face Search (1:N) | ✓ | ✓ |
| Age Estimation | ✓ | ✓ (age range) |
| Gender Detection | ✓ | ✓ |
| Emotion Recognition | ✓ (7 emotions) | ✓ (7 emotions) |
| Beauty Score | ✓ | ✗ |
| 3D Face Modeling | ✓ (advanced) | ✗ |
| Celebrity Recognition | Limited | ✓ (10,000+) |
| Video Analysis | Limited | ✓ (extensive) |
| Free Tier | ✓ (limited) | ✓ (5,000/month, 12 months) |
| Pricing (1M images) | ~$100-300/day tier | $1,000/month |
| Global Infrastructure | Strong in Asia | AWS global |
| On-premise | ✓ (enterprise) | Limited (Panorama) |
| Privacy/Compliance | China-based | US-based (AWS) |
When to Choose Face++#
Choose Face++ if you need:
- Dense landmarks (83-106 points)
- Beauty score analysis
- 3D face modeling (advanced tier)
- Asia-Pacific deployment (strong regional presence)
- Comprehensive attributes (more detailed than AWS)
- On-premise deployment (enterprise SDK)
When to Choose Amazon Rekognition#
Choose Amazon Rekognition if you need:
- AWS ecosystem integration (Lambda, S3, CloudWatch)
- Enterprise-grade reliability (AWS SLA)
- Video analysis (real-time streaming, batch)
- Celebrity recognition (10,000+ celebrities)
- Global deployment (AWS regions worldwide)
- Compliance requirements (SOC, HIPAA, etc.)
- Transparent pricing (clear pay-per-use)
- Free tier ($200 credits, 5,000 images/month)
Commercial APIs vs Self-hosted: Trade-offs#
Advantages of Commercial APIs#
- No infrastructure management: Zero DevOps overhead
- Automatic updates: Models improve without user action
- Scalability: Handle traffic spikes automatically
- Quick start: Minutes to first API call
- Comprehensive features: Age, gender, emotion out-of-the-box
- Support: Professional support teams
Disadvantages of Commercial APIs#
- Cost at scale: High-volume usage expensive ($1,000+/month)
- Network latency: 100-500 ms per call
- Privacy concerns: Data sent to third-party servers
- Vendor lock-in: Proprietary APIs
- Internet dependency: Offline use impossible
- Data residency: Compliance challenges (GDPR, regional laws)
- Rate limits: Throttling on free/low tiers
When to Choose Commercial APIs#
- Startups/MVPs: Quick validation, no infrastructure
- Low-medium volume:
<100,000 faces/month - Cloud-first: Already using cloud services
- Comprehensive attributes: Need age/gender/emotion
- No ML expertise: Managed service
When to Choose Self-hosted (MediaPipe, Dlib, InsightFace)#
- High volume: Millions of faces/month (cost savings)
- Low latency: Sub-50 ms requirements
- Privacy-critical: Healthcare, government, EU
- Offline use: Edge devices, no internet
- Full control: Custom models, fine-tuning
- Long-term cost: Cheaper at scale
Last Updated: January 2025
Dlib Face Detection & Recognition#
1. Overview#
What it is: Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating complex software. Includes highly accurate face detection, facial landmark detection (68-point), and face recognition capabilities.
Maintainer: Davis King (independent developer with community contributions)
License: Boost Software License (open source, commercial-friendly, permissive)
Primary Language: C++ with Python bindings
Active Development Status:
- Repository: https://github.com/davisking/dlib
- Last updated: Actively maintained (2024-2025)
- GitHub stars: 13,000+
- Status: Mature, stable, widely used in academia and industry
2. Core Capabilities#
Face Detection#
- HOG + Linear SVM: Fast, CPU-efficient traditional method
- CNN (MMOD): Highly accurate deep learning detector
- Multi-face detection support
- Both methods provide bounding boxes
Facial Landmarks#
- 68-point model: Industry-standard (iBUG 300-W trained)
- 5-point model: Lightweight alternative for alignment
- 2D landmarks only (no 3D)
- Covers eyes, eyebrows, nose, mouth, jawline
Face Recognition/Identification#
- ResNet-based embedding: 128-dimensional face vectors
- 99.38% accuracy on LFW (with 100x jittering)
- 99.13% accuracy (standard mode)
- One-shot learning capable
- Distance-based similarity matching
Face Attributes#
- Not built-in
- Landmarks can be used to infer head pose, eye closure
3D Face Reconstruction#
- Not supported (2D landmarks only)
Real-time Performance#
- HOG detector: Real-time on CPU (30+ FPS)
- CNN detector: Real-time with GPU, slower on CPU (1-5 FPS)
- Recognition: Fast embedding extraction (
<100ms per face)
3. Technical Architecture#
Underlying Models#
Face Detection#
HOG + Linear SVM
- Histogram of Oriented Gradients feature extraction
- Linear classifier with sliding window
- Image pyramid for multi-scale detection
- Minimum face size: 80x80 pixels
MMOD CNN
- Max-Margin Object Detection
- Custom CNN architecture
- Trained on wide variety of angles and conditions
- Robust to rotation and occlusion
Facial Landmarks#
- 68-point detector: Ensemble of Regression Trees (ERT)
- Based on “One Millisecond Face Alignment” (Kazemi & Sullivan, CVPR 2014)
- Cascade of regressors
- Trained on iBUG 300-W dataset
Face Recognition#
- ResNet-34 architecture
- Trained on ~3 million faces
- 128-dimensional embedding space
- Metric learning with triplet loss
Pre-trained Models#
- shape_predictor_68_face_landmarks.dat: 99.7 MB
- shape_predictor_5_face_landmarks.dat: 9.2 MB (10x smaller)
- mmod_human_face_detector.dat: CNN face detector
- dlib_face_recognition_resnet_model_v1.dat: Face recognition model
- All models downloadable from http://dlib.net/files/
Custom Training#
- Supported: Yes, full training pipeline available
- Object detector trainer: For custom face detection
- Shape predictor trainer: For custom landmark configurations
- DNN training: Complete deep learning training framework
- Documentation: Extensive C++ and Python examples
Model Size#
- HOG detector: Built-in, minimal memory
- CNN detector: ~1-2 MB
- 68-point landmarks: 99.7 MB
- 5-point landmarks: 9.2 MB
- Face recognition: ~25 MB
Dependencies#
- Core: C++ standard library, BLAS/LAPACK (for speed)
- Python bindings: NumPy
- Optional GPU: CUDA (for CNN detector and training)
- No deep learning framework required: Dlib has its own DNN module
4. Performance Benchmarks#
Detection Accuracy#
- HOG: Good for frontal faces, struggles with rotation
- CNN (MMOD): Superior accuracy, handles varied orientations
- Robust to lighting variations (both methods)
- CNN handles occlusions better than HOG
Landmark Accuracy#
- 68-point: 3.78 mean error on 300W benchmark
- Industry-standard, widely validated
- Reliable across diverse datasets
Face Recognition Accuracy#
- LFW benchmark: 99.13% (standard), 99.38% (with jittering)
- Threshold: 0.6 for matching (Euclidean distance)
- State-of-the-art for 2016-2018 era models
- Still competitive in 2024
Speed#
| Method | Device | Performance |
|---|---|---|
| HOG detector | CPU | 30+ FPS |
| CNN detector | CPU | 1-3 FPS |
| CNN detector | GPU | 50+ FPS |
| 68-point landmarks | CPU | 100+ FPS |
| 5-point landmarks | CPU | 110+ FPS (8-10% faster) |
| Face recognition | CPU | 10-50 ms per face |
Latency#
- HOG detection:
<30ms per frame (CPU) - CNN detection: 300-1000 ms (CPU), 20-50 ms (GPU)
- Landmark detection:
<10ms per face - Face encoding: 20-100 ms per face (CPU)
Resource Requirements#
- RAM: 100-200 MB (loaded models)
- GPU memory: 500 MB - 2 GB (for CNN training/inference)
- CPU: Efficient, uses all cores
- Disk: ~150 MB (all models)
5. Platform Support#
Desktop#
- Windows: ✓ (C++, Python)
- macOS: ✓ (C++, Python)
- Linux: ✓ (C++, Python)
Mobile#
- iOS: Possible (C++ integration, unofficial)
- Android: Possible (C++ integration, unofficial)
- Not officially optimized for mobile
Web#
- WebAssembly: Experimental, not official
- Not recommended for browser use
Edge Devices#
- Raspberry Pi: ✓ (HOG detector works well, CNN slow without GPU)
- Embedded Linux: ✓ (C++ lightweight)
Cloud#
- Easily deployed in cloud environments
- Docker-friendly
6. API & Usability#
Python API Quality#
Rating: 8/10
- Clean, Pythonic API
- Well-designed object-oriented interface
- Good documentation
- Some C++ heritage shows through
Code Example: HOG Face Detection#
import dlib
import cv2
# Load the HOG-based face detector
detector = dlib.get_frontal_face_detector()
# Read image
image = cv2.imread('photo.jpg')
# Convert to RGB (dlib uses RGB)
rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Detect faces
# Second parameter: upsample image 1 time (increase for smaller faces)
faces = detector(rgb, 1)
# Draw rectangles
for face in faces:
x1, y1, x2, y2 = face.left(), face.top(), face.right(), face.bottom()
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
print(f"Face detected at: ({x1}, {y1}), ({x2}, {y2})")
cv2.imshow('Face Detection', image)
cv2.waitKey(0)Code Example: CNN Face Detection#
import dlib
import cv2
# Load the CNN face detector
cnn_detector = dlib.cnn_face_detection_model_v1('mmod_human_face_detector.dat')
# Read image
image = cv2.imread('photo.jpg')
rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Detect faces (returns list of mmod_rectangles)
faces = cnn_detector(rgb, 1) # 1 = upsample once
# Draw rectangles
for face in faces:
# face.rect contains the bounding box
x1, y1, x2, y2 = (face.rect.left(), face.rect.top(),
face.rect.right(), face.rect.bottom())
confidence = face.confidence
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
print(f"Face detected with confidence: {confidence:.2f}")
cv2.imshow('CNN Face Detection', image)
cv2.waitKey(0)Code Example: 68-Point Facial Landmarks#
import dlib
import cv2
# Load face detector and shape predictor
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
# Read image
image = cv2.imread('photo.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Detect faces
faces = detector(gray, 1)
# For each face, detect landmarks
for face in faces:
landmarks = predictor(gray, face)
# Draw landmarks
for n in range(68):
x = landmarks.part(n).x
y = landmarks.part(n).y
cv2.circle(image, (x, y), 2, (0, 255, 0), -1)
# Landmark groups:
# 0-16: Jawline
# 17-21: Right eyebrow
# 22-26: Left eyebrow
# 27-35: Nose
# 36-41: Right eye
# 42-47: Left eye
# 48-67: Mouth
cv2.imshow('Facial Landmarks', image)
cv2.waitKey(0)Code Example: Face Recognition#
import dlib
import cv2
import numpy as np
# Load models
detector = dlib.get_frontal_face_detector()
sp = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
facerec = dlib.face_recognition_model_v1('dlib_face_recognition_resnet_model_v1.dat')
def get_face_encoding(image_path):
"""Extract 128D face embedding"""
img = dlib.load_rgb_image(image_path)
faces = detector(img, 1)
if len(faces) == 0:
return None
# Get landmarks and compute face descriptor
shape = sp(img, faces[0])
face_descriptor = facerec.compute_face_descriptor(img, shape)
# Convert to numpy array
return np.array(face_descriptor)
# Compare two faces
encoding1 = get_face_encoding('person1.jpg')
encoding2 = get_face_encoding('person2.jpg')
if encoding1 is not None and encoding2 is not None:
# Compute Euclidean distance
distance = np.linalg.norm(encoding1 - encoding2)
# Threshold: typically 0.6 (lower = more similar)
if distance < 0.6:
print(f"Same person! Distance: {distance:.2f}")
else:
print(f"Different people. Distance: {distance:.2f}")Learning Curve#
Rating: Intermediate (3/5 difficulty)
- Requires model file management (download separately)
- Good documentation but less hand-holding than MediaPipe
- C++ documentation more extensive than Python
- Understanding of traditional CV concepts helpful
Documentation Quality#
Rating: 8/10
- Excellent C++ documentation
- Good Python examples
- Active mailing list and community
- Official site: http://dlib.net
- GitHub: https://github.com/davisking/dlib
7. Pricing/Cost Model#
Free and Open Source
- Boost Software License (very permissive)
- No usage fees
- Commercial use permitted without restrictions
- No attribution required (though appreciated)
- Self-hosted only
8. Integration Ecosystem#
Works With#
- OpenCV: Common pairing for image I/O and preprocessing
- NumPy: Direct conversion to numpy arrays
- Scikit-learn: For building recognition pipelines
- Face_recognition library: High-level wrapper around dlib
- Any C++ project: Native C++ integration
Output Format#
- Detections: Rectangle objects (left, top, right, bottom)
- Landmarks: 68 (x, y) point objects
- Face encodings: 128-dimensional numpy array
- Format: Python objects, easily serialized to JSON
Preprocessing Requirements#
- Color format: RGB (convert from BGR if using OpenCV)
- Minimum face size: 80x80 pixels (default)
- No special normalization: Handles internally
- Alignment recommended: For face recognition, align faces using landmarks
9. Use Case Fit#
Best For#
- Face recognition systems: Security, authentication, photo organization
- 68-point landmarks: Standard facial analysis, expression detection
- Desktop applications: Server-side processing, batch photo analysis
- Research: Well-validated models, reproducible results
- Python projects: Simple integration, no complex dependencies
- C++ applications: Native performance, no overhead
- Custom training: Full control over model training
- Offline processing: No cloud dependency
Ideal Scenarios#
- Photo library face tagging (clustering, search by person)
- Access control systems (door unlock, attendance)
- Batch face analysis (processing archives)
- Research prototyping (academic papers, benchmarks)
- Face alignment preprocessing (for other models)
- Traditional CV pipelines (HOG detector is battle-tested)
Limitations#
- No 3D mesh: Only 2D landmarks (68 points)
- Mobile performance: Not optimized, large model files
- CNN detector slow on CPU: Requires GPU for real-time
- No face attributes: Age, gender, emotion not provided
- Minimum face size: Struggles with very small faces (
<80px) - HOG rotation sensitivity: Frontal faces only with HOG
- Manual model management: Must download .dat files separately
10. Comparison Factors#
Accuracy vs Speed#
- HOG: Fast (30+ FPS CPU) but limited to frontal faces
- CNN: Highly accurate but slow on CPU (1-3 FPS)
- Trade-off: Choose HOG for speed, CNN for accuracy
- Face recognition: Excellent accuracy (99.38% LFW)
Self-hosted vs API#
- Self-hosted only: No official cloud API
- Advantage: No network latency, complete privacy
- Easy deployment: Lightweight, few dependencies
Landmark Quality#
- 68 points: Industry standard since 2014
- Sufficient for: Face alignment, expression analysis, AR filters
- Less detailed than: MediaPipe (468 points), but faster
- More detailed than: MTCNN (5 points), OpenCV (none)
3D Capability#
- No 3D support: 2D landmarks only
- Use instead: MediaPipe or 3D Morphable Models
Privacy#
- On-device processing: Complete privacy
- No telemetry: No data collection
- GDPR-compliant: Ideal for privacy-sensitive applications
Summary Table#
| Feature | Rating/Value |
|---|---|
| Detection Accuracy (HOG) | Good (frontal faces) |
| Detection Accuracy (CNN) | Excellent (all angles) |
| Face Recognition (LFW) | 99.38% |
| Landmark Count | 68 points (2D) |
| Speed (HOG, CPU) | 30+ FPS |
| Speed (CNN, CPU) | 1-3 FPS |
| Speed (CNN, GPU) | 50+ FPS |
| Model Size | ~150 MB (all models) |
| Learning Curve | Intermediate |
| Platform Support | Desktop (excellent), Mobile (limited) |
| Cost | Free (Boost License) |
| 3D Support | No |
| Privacy | On-device (excellent) |
When to Choose Dlib#
Choose Dlib if you need:
- Face recognition/identification (99.38% LFW accuracy)
- 68-point landmarks (industry standard, widely compatible)
- Desktop/server processing (not mobile-first)
- C++ integration (native performance)
- Custom training (full control over models)
- Mature, stable library (10+ years of development)
- Fast CPU detection (HOG detector, 30+ FPS)
- Research-validated models (reproducible benchmarks)
Avoid Dlib if you need:
- 3D face mesh (use MediaPipe)
- Real-time mobile performance (use MediaPipe)
- Dense landmarks
>68points (use MediaPipe) - Face attributes like age/gender (use commercial APIs)
- Extremely fast detection on all angles (use RetinaFace + GPU)
- No model file management (use cloud APIs)
Last Updated: January 2025
InsightFace: 2D & 3D Face Analysis#
1. Overview#
What it is: InsightFace is a state-of-the-art open-source 2D and 3D face analysis toolkit. Known for industry-leading face recognition (ArcFace method), face detection, face alignment, and face attribute analysis. Production-ready with ONNX Runtime support.
Maintainer: Jia Guo and Jiankang Deng (DeepInsight team, originally Megvii/Face++)
License: Mixed:
- Non-commercial research: Free
- Commercial use: Requires separate license (contact team)
- Models: Various licenses per model
Primary Language: Python (primary), with C++ support via ONNX Runtime
Active Development Status:
- Repository: https://github.com/deepinsight/insightface
- Last updated: Actively maintained (2024-2025)
- GitHub stars: 23,000+
- Status: Production-ready, widely used in industry
2. Core Capabilities#
Face Detection#
- RetinaFace: High-accuracy single-stage detector
- SCRFD: Efficient face detection (Sample and Computation Redistribution)
- Multi-scale detection
- Facial landmark output (5 points) with detection
Facial Landmarks#
- 5-point landmarks: Eyes, nose, mouth corners (with detection)
- 106-point landmarks: Dense 2D landmarks (optional model)
- 3D landmarks: 68-point 3D landmarks via 3D reconstruction models
- Integrated with detection pipeline
Face Recognition/Identification#
- ArcFace: State-of-the-art recognition method (99.83% LFW)
- Multiple backbones: iResNet, MobileFaceNet, others
- 128-512D embeddings: Configurable vector size
- One-shot and few-shot learning
- Partial face recognition
- Masked face recognition: Trained on occluded faces
Face Attributes#
- Age estimation
- Gender classification
- Face quality assessment
- Pose estimation (yaw, pitch, roll)
3D Face Reconstruction#
- Yes: Full 3D face reconstruction models available
- 3D alignment
- 3D shape and texture extraction
Real-time Performance#
- Optimized for real-time: 30+ FPS with efficient models
- ONNX Runtime enables GPU/CPU acceleration
- Mobile-friendly models available (MobileFaceNet)
3. Technical Architecture#
Underlying Models#
Face Detection#
RetinaFace: Single-stage detector with multi-task learning
- Backbone: ResNet, MobileNet variants
- Detects faces + 5 landmarks simultaneously
- Multi-scale pyramid network
SCRFD: Efficient detection
- Sample and Computation Redistribution
- Faster than RetinaFace with comparable accuracy
- Optimized for edge devices
Face Recognition#
ArcFace (Additive Angular Margin Loss)
- Backbone: iResNet (improved ResNet) - ResNet34, 50, 100
- Trained on large-scale datasets (MS1MV2, MS1MV3, WebFace)
- Metric learning with angular margin
- 512D embeddings (standard)
Alternative methods: CosFace, Combined Margin, SphereFace
Landmark Detection#
- Integrated with detection models
- Separate dense landmark models available
Pre-trained Models#
- buffalo_l: General-purpose, balanced accuracy/speed
- buffalo_sc: High accuracy, larger model
- antelopev2: Latest model pack
- Models available: Detection, recognition, alignment, attributes, 3D reconstruction
- Model zoo: https://github.com/deepinsight/insightface/tree/master/model_zoo
Custom Training#
- Fully supported: Complete training pipelines
- ArcFace training: PyTorch implementation available
- Detection training: RetinaFace, SCRFD training code
- Datasets: Tools for dataset preparation
- Documentation: Extensive training guides
Model Size#
- Detection models: 1-10 MB (depending on backbone)
- Recognition models: 100-300 MB (ResNet-based), 5-15 MB (MobileNet)
- Total typical deployment: 50-200 MB
- Lightweight options: Sub-10 MB for mobile
Dependencies#
- ONNX Runtime: Primary inference engine
- onnxruntime-gpu or onnxruntime (CPU)
- NumPy: Data handling
- OpenCV: Image preprocessing (optional)
- Training: PyTorch 1.12+, MXNet (legacy)
- No TensorFlow required
4. Performance Benchmarks#
Detection Accuracy#
- RetinaFace ResNet-50: 96.3% (easy), 95.6% (medium), 91.4% (hard) on WIDER FACE
- SCRFD: Comparable to RetinaFace with better speed
- State-of-the-art on WIDER FACE benchmark
Face Recognition Accuracy#
- ArcFace on LFW: 99.83% (top-tier)
- IJB-B: 96.21% at FAR=1e-4
- IJB-C: 97.37% at FAR=1e-4
- AgeDB-30: 98.15%
- CFP-FP: 99.08%
- buffalo_l model: 99.88% detection success on LFW
Comparison with Competitors#
- ArcFace: 99.83% LFW
- CosineFace: 99.80% LFW
- SphereFace: 99.76% LFW
- Dlib: 99.38% LFW
- InsightFace consistently top-5 on NIST-FRVT 1:1 leaderboard
Speed#
| Model | Device | Performance |
|---|---|---|
| SCRFD-0.5GF | CPU | 100+ FPS |
| SCRFD-10GF | GPU | 200+ FPS |
| RetinaFace (MobileNet) | GPU | 60+ FPS |
| ArcFace (iResNet100) | GPU | 30-50 ms per face |
| MobileFaceNet | CPU | 10-20 ms per face |
Latency#
- Detection: 10-30 ms per image (GPU)
- Recognition embedding: 20-100 ms per face (depending on model)
- End-to-end: 50-150 ms per face (detection + recognition)
Resource Requirements#
- RAM: 200-500 MB (loaded models)
- GPU memory: 1-4 GB (depending on batch size)
- CPU: Multi-threaded, efficient
- Disk: 100-300 MB (typical deployment)
5. Platform Support#
Desktop#
- Windows: ✓ (Python, ONNX)
- macOS: ✓ (Python, ONNX)
- Linux: ✓ (Python, ONNX, primary platform)
Mobile#
- iOS: ✓ (ONNX Runtime, CoreML conversion)
- Android: ✓ (ONNX Runtime, TFLite conversion)
- MobileFaceNet optimized for mobile
Web#
- JavaScript/WebAssembly: Possible via ONNX.js
- Not officially supported
- Requires conversion and optimization
Edge Devices#
- Raspberry Pi: ✓ (lightweight models)
- Jetson Nano/Xavier: ✓ (excellent with GPU)
- NVIDIA devices: First-class support
Cloud#
- Easily deployed in cloud (Docker, Kubernetes)
- ONNX Runtime cloud-friendly
6. API & Usability#
Python API Quality#
Rating: 9/10
- Clean, modern Python API
- Well-structured model zoo
- Easy model loading and inference
- Good abstraction over ONNX complexity
Code Example: Face Detection and Recognition#
import cv2
import numpy as np
from insightface.app import FaceAnalysis
# Initialize the face analysis app
app = FaceAnalysis(name='buffalo_l', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))
# Read image
image = cv2.imread('photo.jpg')
# Detect and analyze faces
faces = app.get(image)
# Iterate through detected faces
for face in faces:
# Bounding box
bbox = face.bbox.astype(int)
print(f"Face detected at: {bbox}")
# Draw bounding box
cv2.rectangle(image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (0, 255, 0), 2)
# 5-point landmarks
landmarks = face.kps.astype(int)
for point in landmarks:
cv2.circle(image, tuple(point), 2, (0, 0, 255), -1)
# Face embedding (512D vector)
embedding = face.embedding
print(f"Embedding shape: {embedding.shape}") # (512,)
# Face attributes
if hasattr(face, 'age'):
print(f"Age: {face.age}")
if hasattr(face, 'gender'):
gender = 'Male' if face.gender == 1 else 'Female'
print(f"Gender: {gender}")
# Face quality score
if hasattr(face, 'det_score'):
print(f"Detection confidence: {face.det_score:.2f}")
cv2.imshow('Face Analysis', image)
cv2.waitKey(0)Code Example: Face Comparison (1:1 Verification)#
import cv2
import numpy as np
from insightface.app import FaceAnalysis
# Initialize
app = FaceAnalysis(name='buffalo_l')
app.prepare(ctx_id=0)
def get_face_embedding(image_path):
"""Extract face embedding from image"""
img = cv2.imread(image_path)
faces = app.get(img)
if len(faces) == 0:
return None
# Return embedding of first face
return faces[0].embedding
# Compare two faces
embedding1 = get_face_embedding('person1.jpg')
embedding2 = get_face_embedding('person2.jpg')
if embedding1 is not None and embedding2 is not None:
# Compute cosine similarity
similarity = np.dot(embedding1, embedding2) / (
np.linalg.norm(embedding1) * np.linalg.norm(embedding2)
)
# Threshold: typically 0.3-0.5 for ArcFace (higher = more similar)
threshold = 0.4
if similarity > threshold:
print(f"Same person! Similarity: {similarity:.3f}")
else:
print(f"Different people. Similarity: {similarity:.3f}")Code Example: Custom Model Loading#
import cv2
from insightface.model_zoo import get_model
# Load specific detection model
detector = get_model('retinaface_r50_v1', providers=['CUDAExecutionProvider'])
detector.prepare(ctx_id=0, input_size=(640, 640))
# Load specific recognition model
recognizer = get_model('arcface_r100_v1', providers=['CUDAExecutionProvider'])
recognizer.prepare(ctx_id=0)
# Read image
image = cv2.imread('photo.jpg')
# Detect faces
bboxes, landmarks = detector.detect(image)
# Extract embeddings
for i, bbox in enumerate(bboxes):
# Align face using landmarks
aligned_face = recognizer.get_aligned_face(image, landmarks[i])
# Get embedding
embedding = recognizer.get_embedding(aligned_face)
print(f"Face {i} embedding: {embedding.shape}")Code Example: Face Search (1:N Identification)#
import numpy as np
import cv2
from insightface.app import FaceAnalysis
# Initialize
app = FaceAnalysis(name='buffalo_l')
app.prepare(ctx_id=0)
# Build database of known faces
database = {}
def register_face(name, image_path):
"""Add face to database"""
img = cv2.imread(image_path)
faces = app.get(img)
if len(faces) > 0:
database[name] = faces[0].embedding
print(f"Registered: {name}")
# Register known faces
register_face("Alice", "alice.jpg")
register_face("Bob", "bob.jpg")
register_face("Charlie", "charlie.jpg")
def search_face(query_image_path, threshold=0.4):
"""Find matching face in database"""
img = cv2.imread(query_image_path)
faces = app.get(img)
if len(faces) == 0:
return None, 0.0
query_embedding = faces[0].embedding
# Compare with all database embeddings
best_match = None
best_similarity = 0.0
for name, db_embedding in database.items():
similarity = np.dot(query_embedding, db_embedding) / (
np.linalg.norm(query_embedding) * np.linalg.norm(db_embedding)
)
if similarity > best_similarity:
best_similarity = similarity
best_match = name
if best_similarity > threshold:
return best_match, best_similarity
else:
return "Unknown", best_similarity
# Search for face
name, score = search_face("query.jpg")
print(f"Matched: {name} (confidence: {score:.3f})")Learning Curve#
Rating: Intermediate (3/5 difficulty)
- Simple API for basic use
- Model zoo structure requires understanding
- ONNX Runtime setup can be tricky (GPU drivers)
- Advanced features need deeper knowledge
- Good examples available
Documentation Quality#
Rating: 8/10
- Comprehensive GitHub documentation
- Good model zoo descriptions
- Training guides available
- Community support on GitHub Issues
- Some Chinese-language resources
- Links:
7. Pricing/Cost Model#
Open Source with Commercial Considerations
- Research/Non-commercial: Free
- Commercial use: Contact team for licensing
- Model licenses: Vary by model (check model zoo)
- No API fees: Self-hosted only
- Training code: Freely available
8. Integration Ecosystem#
Works With#
- ONNX Runtime: Primary inference engine
- OpenCV: Image I/O and preprocessing
- NumPy: Embedding manipulation
- PyTorch: Training pipelines
- MXNet: Legacy training (older versions)
- TensorRT: NVIDIA optimization
- CoreML: iOS deployment
- TFLite: Android optimization
Output Format#
- Detections: Bounding boxes (x1, y1, x2, y2), confidence scores
- Landmarks: 5-point (eyes, nose, mouth) or 106-point arrays
- Embeddings: NumPy arrays (512D or 128D)
- Attributes: Age (int), gender (binary), pose (yaw/pitch/roll)
- Format: Python objects, easily serialized to JSON
Preprocessing Requirements#
- Input: BGR images (OpenCV format) or RGB
- Resolution: Flexible (models handle scaling)
- Alignment: Handled internally for recognition
- Normalization: Automatic
9. Use Case Fit#
Best For#
- Face recognition systems: Industry-leading accuracy (99.83% LFW)
- Security and surveillance: High accuracy, handles occlusions
- Photo organization: Face clustering, search by person
- Access control: Authentication, attendance systems
- Social media: Face tagging, verification
- Research: State-of-the-art benchmarks, reproducible
- Production deployments: ONNX Runtime stability, cross-platform
- Masked face recognition: Models trained on occluded faces
Ideal Scenarios#
- Large-scale face databases (millions of identities)
- Banking/fintech KYC (Know Your Customer) verification
- Airport security and border control
- Photo album auto-tagging (Google Photos style)
- Video surveillance analytics
- Attendance tracking in schools/offices
- Age verification systems
- Celebrity/VIP recognition
Limitations#
- Commercial licensing: Requires permission for commercial use
- No 468-point mesh: Less detailed than MediaPipe for AR
- Model size: Larger than MediaPipe for high accuracy
- GPU recommended: CPU performance acceptable but slower
- ONNX Runtime dependency: Additional setup complexity
- No face attributes in all models: Age/gender require specific models
10. Comparison Factors#
Accuracy vs Speed#
- Highest accuracy: 99.83% on LFW (beats Dlib, MediaPipe for recognition)
- Flexible speed: Lightweight models (SCRFD) to high-accuracy (RetinaFace)
- Sweet spot: Best accuracy-speed trade-off for recognition
- GPU-optimized: Excellent performance with GPU
Self-hosted vs API#
- Self-hosted only: No official cloud API
- Advantage: No per-call costs, privacy, control
- ONNX portability: Deploy anywhere
Landmark Quality#
- 5 points (standard): Basic alignment, fast
- 106 points (optional): More detailed than Dlib 68
- Less than MediaPipe: 468 points vs 106/5
- Sufficient for recognition: 5 points adequate for alignment
3D Capability#
- Yes: 3D reconstruction models available
- 3D alignment: Supported
- Not as detailed: Less than MediaPipe’s 3D mesh
Privacy#
- On-device processing: Complete privacy
- No cloud dependency: GDPR-compliant
- Self-hosted: Full control over data
Summary Table#
| Feature | Rating/Value |
|---|---|
| Detection Accuracy (WIDER FACE) | 91.4% (hard), 96.3% (easy) |
| Face Recognition (LFW) | 99.83% (ArcFace) |
| Landmark Count | 5 points (standard), 106 (optional) |
| Speed (GPU, detection) | 60-200+ FPS |
| Speed (GPU, recognition) | 30-50 ms per face |
| Model Size | 50-200 MB (typical) |
| Learning Curve | Intermediate |
| Platform Support | Excellent (desktop, mobile via ONNX) |
| Cost | Free (non-commercial), License (commercial) |
| 3D Support | Yes (3D reconstruction models) |
| Privacy | On-device (excellent) |
When to Choose InsightFace#
Choose InsightFace if you need:
- Highest face recognition accuracy (99.83% LFW, industry-leading)
- Production-grade recognition (security, banking, surveillance)
- Large-scale face databases (millions of identities)
- State-of-the-art models (ArcFace, RetinaFace, SCRFD)
- Flexible deployment (ONNX Runtime, cross-platform)
- Masked face recognition (occluded faces, COVID-era use cases)
- Research benchmarks (reproducible SOTA results)
- Custom training (full training pipelines available)
Avoid InsightFace if you need:
- Dense 3D mesh (468 points) for AR effects (use MediaPipe)
- Simple 68-point landmarks without recognition (use Dlib)
- Commercial use without licensing (use MediaPipe, Dlib)
- Minimal setup complexity (use cloud APIs like AWS Rekognition)
- Web-first deployment (use MediaPipe JavaScript)
Last Updated: January 2025
MediaPipe Face Detection & Mesh#
1. Overview#
What it is: MediaPipe Face is Google’s open-source framework for real-time face detection, facial landmarks, and 3D face mesh estimation. Part of the broader MediaPipe ecosystem for cross-platform ML solutions.
Maintainer: Google AI Edge Team (formerly Google Research)
License: Apache 2.0 (open source, commercial-friendly)
Primary Language: C++ core with Python, JavaScript, and mobile SDKs
Active Development Status:
- Repository: https://github.com/google-ai-edge/mediapipe
- Last updated: Actively maintained (2024-2025)
- GitHub stars: 26,000+
- Status: Production-ready, widely deployed
2. Core Capabilities#
Face Detection#
- BlazeFace detector: Optimized for mobile devices, detects faces in full images
- Bounding box detection with confidence scores
- Multi-face detection support
Facial Landmarks#
- 468-point 3D face mesh: Industry-leading landmark density
- Real-time 3D surface geometry estimation
- Includes eye regions (71 landmarks), lips (80 landmarks), face oval (36 landmarks)
- Optional attention mesh for iris tracking (5 landmarks per eye)
Face Recognition/Identification#
- Not built-in (detection and landmarks only)
- Can be used as preprocessing for recognition pipelines
Face Attributes#
- Not directly provided
- Landmarks can be used to infer attributes (mouth open, eye closure, head pose)
3D Face Reconstruction#
- Full 3D mesh: 468 vertices with UV coordinates
- Face geometry estimation from single RGB camera
- No depth sensor required
Real-time Performance#
- Designed for real-time: 30-100+ FPS on mobile devices
- Optimized for both CPU and GPU
3. Technical Architecture#
Underlying Models#
- BlazeFace: SSD-based face detector (MobileNetV2 backbone)
- Face Mesh: Custom CNN for landmark regression
- Two-stage pipeline: detection → landmark estimation
Pre-trained Models#
- Face Detection (short-range): Optimized for faces within 2 meters
- Face Detection (full-range): Handles faces at greater distances
- Face Mesh: Single model with 468 landmarks
- Face Mesh with Attention: Includes iris tracking
Custom Training#
- Models are pre-trained and frozen
- Not designed for custom training
- Source code available but requires expertise to retrain
Model Size#
- Face Detection: ~1-3 MB
- Face Mesh: ~3-5 MB
- Total pipeline:
<10MB (very lightweight)
Dependencies#
- Standalone: MediaPipe includes all dependencies
- Optional GPU: OpenGL ES 3.0+, Metal (iOS), or OpenGL (desktop)
- Python: NumPy, OpenCV (for I/O only)
- No TensorFlow or PyTorch required for inference
4. Performance Benchmarks#
Detection Accuracy#
- MediaPipe vs competitors: 99.3% accuracy (comparative study)
- 300W benchmark: 3.12 mean error (better than Dlib’s 3.78)
- State-of-the-art for mobile and embedded devices
Landmark Accuracy#
- 468 3D landmarks with sub-pixel accuracy
- Robust to occlusions, lighting variations, and head poses
- Superior to traditional 68-point detectors for dense mesh applications
Speed#
- Mobile (CPU): 30-60 FPS on modern smartphones
- Desktop (CPU): 60-100+ FPS
- GPU acceleration: 100+ FPS on modest GPUs
- Embedded (Raspberry Pi): 9-13 FPS (CPU only)
Latency#
- Detection: 5-10 ms per frame (desktop CPU)
- Full mesh: 15-30 ms per frame (desktop CPU)
- Lower latency on GPU
Resource Requirements#
- RAM: 50-100 MB
- GPU memory: Minimal (
<100MB) - Model size:
<10MB total - CPU: Runs on mobile processors, optimized for ARM
5. Platform Support#
Desktop#
- Windows: ✓ (Python, C++)
- macOS: ✓ (Python, C++)
- Linux: ✓ (Python, C++)
Mobile#
- iOS: ✓ (Objective-C, Swift)
- Android: ✓ (Java, Kotlin)
Web#
- JavaScript/WebAssembly: ✓ (TensorFlow.js-based)
- Runs in browser with WebGL acceleration
Edge Devices#
- Raspberry Pi: ✓ (reduced performance)
- Embedded: ✓ (ARM processors, requires optimization)
Cloud#
- Can be deployed in cloud environments (not required)
6. API & Usability#
Python API Quality#
Rating: 9/10
- Clean, intuitive API
- Well-documented
- Consistent across MediaPipe solutions
Code Example: Simple Face Detection#
import cv2
import mediapipe as mp
# Initialize MediaPipe Face Detection
mp_face_detection = mp.solutions.face_detection
mp_drawing = mp.solutions.drawing_utils
# Create face detection object
with mp_face_detection.FaceDetection(
model_selection=0, # 0 for short-range (< 2m), 1 for full-range
min_detection_confidence=0.5
) as face_detection:
# Read image
image = cv2.imread('photo.jpg')
# Convert BGR to RGB
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Process the image
results = face_detection.process(image_rgb)
# Draw detections
if results.detections:
for detection in results.detections:
mp_drawing.draw_detection(image, detection)
# Get bounding box
bbox = detection.location_data.relative_bounding_box
print(f"Face detected: {bbox.xmin:.2f}, {bbox.ymin:.2f}, "
f"{bbox.width:.2f}, {bbox.height:.2f}")
# Display result
cv2.imshow('Face Detection', image)
cv2.waitKey(0)Code Example: 468-Point Face Mesh#
import cv2
import mediapipe as mp
# Initialize MediaPipe Face Mesh
mp_face_mesh = mp.solutions.face_mesh
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
# Create face mesh object
with mp_face_mesh.FaceMesh(
static_image_mode=True,
max_num_faces=1,
refine_landmarks=True, # Include iris landmarks
min_detection_confidence=0.5,
min_tracking_confidence=0.5
) as face_mesh:
# Read image
image = cv2.imread('photo.jpg')
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Process the image
results = face_mesh.process(image_rgb)
# Draw face landmarks
if results.multi_face_landmarks:
for face_landmarks in results.multi_face_landmarks:
mp_drawing.draw_landmarks(
image=image,
landmark_list=face_landmarks,
connections=mp_face_mesh.FACEMESH_TESSELATION,
landmark_drawing_spec=None,
connection_drawing_spec=mp_drawing_styles
.get_default_face_mesh_tesselation_style()
)
# Access individual landmarks
for idx, landmark in enumerate(face_landmarks.landmark):
# landmark.x, landmark.y, landmark.z (normalized coordinates)
h, w, c = image.shape
x = int(landmark.x * w)
y = int(landmark.y * h)
# Use landmark coordinates for analysis
cv2.imshow('Face Mesh', image)
cv2.waitKey(0)Learning Curve#
Rating: Beginner-friendly (2/5 difficulty)
- Simple API, minimal setup
- Excellent tutorials and examples
- No ML expertise required for basic use
- Advanced customization requires deeper knowledge
Documentation Quality#
Rating: 9/10
- Comprehensive official documentation
- Interactive demos on website
- Active community support
- Code examples for all platforms
- Links:
7. Pricing/Cost Model#
Free and Open Source
- Apache 2.0 license
- No usage fees
- No API calls or rate limits
- Commercial use permitted
- Self-hosted only (no cloud service)
8. Integration Ecosystem#
Works With#
- OpenCV: Common pairing for video I/O
- NumPy: Landmark data as NumPy arrays
- TensorFlow: Can integrate into TF pipelines (optional)
- Unity: Via MediaPipe Unity Plugin
- Unreal Engine: Via custom integration
Output Format#
- Detections: Bounding boxes (normalized coordinates), confidence scores
- Landmarks: 468 3D points (x, y, z normalized), plus visibility and presence scores
- Format: Python objects, easily converted to NumPy arrays, JSON
Preprocessing Requirements#
- Input: RGB images (any resolution)
- Color format: Requires RGB (convert from BGR if using OpenCV)
- No preprocessing: Model handles scaling and normalization internally
9. Use Case Fit#
Best For#
- Real-time applications: Webcam, video streaming, AR filters
- Mobile apps: iOS/Android face tracking, selfie effects
- Dense landmark needs: 468 points for detailed facial analysis
- Cross-platform: Single codebase for mobile, web, desktop
- 3D face modeling: Virtual try-on, AR avatars, mesh-based effects
- Privacy-conscious: On-device processing, no cloud required
- Web applications: Browser-based face tracking (WebAssembly)
Ideal Scenarios#
- Augmented reality filters (Snapchat-style)
- Video conferencing effects (background blur based on face position)
- Emotion analysis (via landmark geometry)
- Gaze tracking (with iris landmarks)
- Photo organization (face detection for albums)
- Accessibility features (head pose for cursor control)
Limitations#
- No face recognition: Doesn’t identify individuals (use with ArcFace/InsightFace)
- No face attributes: Age, gender, emotion not directly provided
- Frozen models: Custom training requires significant effort
- Computational cost: 468 landmarks more expensive than 68-point alternatives
- Minimum face size: Performance degrades on very small faces (
<20px)
10. Comparison Factors#
Accuracy vs Speed#
- High accuracy: State-of-the-art for mobile (99.3%)
- Fast: 30+ FPS on mobile, 100+ FPS on desktop
- Sweet spot: Best balance for real-time mobile applications
Self-hosted vs API#
- Self-hosted only: No cloud API available
- Advantage: No network latency, privacy-friendly
- Disadvantage: Must deploy and maintain locally
Landmark Quality#
- 468-point mesh: Most detailed among open-source solutions
- 3D coordinates: Depth information from single RGB camera
- Superior to: Dlib (68 points), MTCNN (5 points), OpenCV (no landmarks)
- Trade-off: More computational overhead than sparse landmarks
3D Capability#
- Full 3D mesh: Yes, industry-leading
- UV coordinates: Yes, for texture mapping
- Real-time 3D: Yes, optimized pipeline
Privacy#
- On-device processing: Complete privacy, no data leaves device
- No telemetry: No usage tracking or data collection
- GDPR-friendly: Ideal for privacy-sensitive applications
Summary Table#
| Feature | Rating/Value |
|---|---|
| Detection Accuracy | 99.3% (99.3% in comparative study) |
| Landmark Count | 468 points (3D) |
| Speed (Desktop CPU) | 60-100+ FPS |
| Speed (Mobile CPU) | 30-60 FPS |
| Model Size | <10 MB |
| Learning Curve | Beginner-friendly |
| Platform Support | Excellent (mobile, web, desktop) |
| Cost | Free (Apache 2.0) |
| 3D Support | Full 3D mesh |
| Privacy | On-device (excellent) |
When to Choose MediaPipe Face#
Choose MediaPipe Face if you need:
- Dense 3D face mesh (468 landmarks) for AR effects, virtual try-on
- Real-time performance on mobile devices (iOS/Android)
- Cross-platform support (web, mobile, desktop from single codebase)
- On-device privacy (no cloud, GDPR-compliant)
- Lightweight models (
<10MB) for app size constraints - Google-backed stability for production applications
Avoid MediaPipe if you need:
- Face recognition/identification (use InsightFace, Dlib)
- Face attributes like age, gender, emotion (use commercial APIs)
- Custom training on your dataset (use RetinaFace, PyTorch-based solutions)
- Only basic detection/68 landmarks (Dlib is simpler)
Last Updated: January 2025
MTCNN: Multi-task Cascaded Convolutional Networks#
1. Overview#
What it is: MTCNN is a deep learning-based face detection and alignment method using three cascaded convolutional neural networks to detect faces and facial landmarks. Popular for its balance of accuracy and speed, especially in the mid-2010s era.
Maintainer: Original paper by Kaipeng Zhang et al. (2016), multiple open-source implementations by community
License: Varies by implementation:
- Original paper: Academic research
- Popular implementations: MIT License (ipazc/mtcnn on GitHub)
Primary Language: Python (most implementations), TensorFlow/PyTorch/Caffe backends
Active Development Status:
- Original paper: 2016 (CVPR, ECCV)
- Community implementations: Maintained but less active than newer methods
- Most popular repo: https://github.com/ipazc/mtcnn (5,000+ stars)
- Status: Mature, stable, but surpassed by newer methods (RetinaFace, SCRFD)
2. Core Capabilities#
Face Detection#
- Multi-scale detection: Handles faces of various sizes via image pyramid
- Bounding box regression: Precise face localization
- Three-stage cascade: Coarse-to-fine detection (P-Net → R-Net → O-Net)
- Multi-face detection support
Facial Landmarks#
- 5-point landmarks: Left eye, right eye, nose, left mouth corner, right mouth corner
- Output simultaneously with detection
- Used for face alignment
Face Recognition/Identification#
- Not included (detection and landmarks only)
- Often used as preprocessing for recognition pipelines
Face Attributes#
- Not supported
3D Face Reconstruction#
- Not supported (2D landmarks only)
Real-time Performance#
- Real-time capable: 20-40 FPS on GPU
- Slower on CPU: 5-15 FPS (depending on image size and upsampling)
- Cascade design allows early rejection for efficiency
3. Technical Architecture#
Underlying Models#
Three-Stage Cascade#
P-Net (Proposal Network)
- Lightweight CNN (12x12 receptive field)
- Operates on image pyramid (multiple scales)
- Generates candidate face regions
- Fast, coarse detection
R-Net (Refine Network)
- Deeper CNN (24x24 input)
- Refines proposals from P-Net
- Rejects many false positives
- Bounding box regression
O-Net (Output Network)
- Most complex CNN (48x48 input)
- Final classification and refinement
- Outputs 5 facial landmarks
- Highest accuracy stage
Architecture Details#
- Fully convolutional: Efficient multi-scale processing
- Multi-task learning: Simultaneously predicts face/non-face, bounding box, and landmarks
- Coarse-to-fine: Each stage refines results from previous stage
- Early rejection: Non-faces rejected early, saves computation
Pre-trained Models#
- Models trained on WIDER FACE and CelebA datasets
- Implementations include pre-trained weights
- Models typically bundled with library installation
- No separate download needed for most packages
Custom Training#
- Possible but uncommon: Original training code available
- Datasets needed: Face detection + landmark annotations
- Complexity: Requires training all three networks
- Most users rely on pre-trained models
Model Size#
- P-Net: ~30 KB
- R-Net: ~400 KB
- O-Net: ~1.5 MB
- Total: ~2 MB (very lightweight)
Dependencies#
- TensorFlow (most common implementation) or PyTorch
- OpenCV: Image preprocessing
- NumPy: Array operations
- Lightweight, minimal dependencies
4. Performance Benchmarks#
Detection Accuracy#
- WIDER FACE: Outperformed state-of-the-art at publication (2016)
- FDDB: Superior accuracy to Haar cascades, HOG, early CNNs
- Comparative study: 97.56% AUC (vs R-CNN 91.24%, Faster R-CNN 92.01%)
- Still competitive for frontal faces, but surpassed by modern methods (RetinaFace, SCRFD)
Landmark Accuracy#
- 5-point landmarks: Good accuracy for alignment
- AFLW benchmark: Strong performance in 2016
- Sufficient for face alignment preprocessing
- Less detailed than 68-point (Dlib) or 468-point (MediaPipe)
Speed#
| Configuration | Device | Performance |
|---|---|---|
| Default settings | CPU | 5-10 FPS |
| Optimized settings | CPU | 10-15 FPS |
| GPU acceleration | GPU | 20-40 FPS |
| Large images | CPU | 2-5 FPS |
| Small images (640x480) | CPU | 15-25 FPS |
Speed vs Accuracy Trade-offs#
- Scale factor: Larger = faster, less accurate (typical: 0.709)
- Min face size: Larger = faster (typical: 20-40 pixels)
- Thresholds: Higher = faster, misses some faces
Latency#
- CPU: 100-300 ms per image (depending on size, faces)
- GPU: 25-50 ms per image
- Single face: Faster due to early rejection cascade
Resource Requirements#
- RAM: 50-100 MB
- GPU memory: 500 MB - 1 GB
- Model size: 2 MB (minimal)
- CPU: Multi-threaded, moderate efficiency
5. Platform Support#
Desktop#
- Windows: ✓ (Python)
- macOS: ✓ (Python)
- Linux: ✓ (Python)
Mobile#
- iOS: Possible (TensorFlow Lite conversion)
- Android: Possible (TensorFlow Lite conversion)
- Not officially optimized, but lightweight enough
Web#
- JavaScript: Possible (TensorFlow.js conversion)
- Community implementations exist
- Not recommended vs MediaPipe for web
Edge Devices#
- Raspberry Pi: ✓ (runs acceptably on CPU)
- Embedded: ✓ (small model size is advantage)
- Jetson: ✓ (good performance with GPU)
Cloud#
- Easily deployed in cloud environments
- Docker-friendly
6. API & Usability#
Python API Quality#
Rating: 8/10 (for ipazc/mtcnn implementation)
- Simple, intuitive API
- Minimal configuration needed
- Good for quick prototyping
- Some implementations better documented than others
Code Example: Basic Face Detection#
from mtcnn import MTCNN
import cv2
# Initialize detector
detector = MTCNN()
# Read image
image = cv2.imread('photo.jpg')
# Convert BGR to RGB (MTCNN expects RGB)
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Detect faces
faces = detector.detect_faces(image_rgb)
# Draw results
for face in faces:
# Bounding box
x, y, width, height = face['box']
cv2.rectangle(image, (x, y), (x + width, y + height), (0, 255, 0), 2)
# Confidence score
confidence = face['confidence']
cv2.putText(image, f'{confidence:.2f}', (x, y - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
# 5-point landmarks
keypoints = face['keypoints']
for key, point in keypoints.items():
cv2.circle(image, point, 2, (0, 0, 255), -1)
# Individual landmarks
left_eye = keypoints['left_eye']
right_eye = keypoints['right_eye']
nose = keypoints['nose']
mouth_left = keypoints['mouth_left']
mouth_right = keypoints['mouth_right']
print(f"Face detected at ({x}, {y}), confidence: {confidence:.2f}")
cv2.imshow('MTCNN Detection', image)
cv2.waitKey(0)Code Example: Custom Parameters#
from mtcnn import MTCNN
import cv2
# Initialize with custom parameters
detector = MTCNN(
min_face_size=40, # Minimum face size to detect (pixels)
steps_threshold=[0.6, 0.7, 0.8], # Thresholds for P-Net, R-Net, O-Net
scale_factor=0.709 # Scale factor for image pyramid
)
# For faster processing (less accurate):
# detector = MTCNN(min_face_size=60, steps_threshold=[0.7, 0.8, 0.9])
# For higher accuracy (slower):
# detector = MTCNN(min_face_size=20, steps_threshold=[0.5, 0.6, 0.7])
image_rgb = cv2.cvtColor(cv2.imread('photo.jpg'), cv2.COLOR_BGR2RGB)
faces = detector.detect_faces(image_rgb)
print(f"Detected {len(faces)} faces")Code Example: Face Alignment#
from mtcnn import MTCNN
import cv2
import numpy as np
detector = MTCNN()
def align_face(image, left_eye, right_eye):
"""Align face based on eye positions"""
# Compute angle between eyes
dx = right_eye[0] - left_eye[0]
dy = right_eye[1] - left_eye[1]
angle = np.degrees(np.arctan2(dy, dx))
# Compute center point between eyes
center = ((left_eye[0] + right_eye[0]) // 2,
(left_eye[1] + right_eye[1]) // 2)
# Get rotation matrix
M = cv2.getRotationMatrix2D(center, angle, scale=1.0)
# Perform affine transformation
aligned = cv2.warpAffine(image, M, (image.shape[1], image.shape[0]))
return aligned
image = cv2.imread('photo.jpg')
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
faces = detector.detect_faces(image_rgb)
if faces:
keypoints = faces[0]['keypoints']
aligned_face = align_face(image, keypoints['left_eye'], keypoints['right_eye'])
cv2.imshow('Original', image)
cv2.imshow('Aligned', aligned_face)
cv2.waitKey(0)Code Example: Batch Processing#
from mtcnn import MTCNN
import cv2
import os
detector = MTCNN()
def process_folder(input_folder, output_folder):
"""Process all images in a folder"""
os.makedirs(output_folder, exist_ok=True)
for filename in os.listdir(input_folder):
if filename.lower().endswith(('.jpg', '.jpeg', '.png')):
# Read image
image_path = os.path.join(input_folder, filename)
image = cv2.imread(image_path)
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Detect faces
faces = detector.detect_faces(image_rgb)
# Draw detections
for face in faces:
x, y, w, h = face['box']
cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)
# Save result
output_path = os.path.join(output_folder, filename)
cv2.imwrite(output_path, image)
print(f"Processed {filename}: {len(faces)} faces detected")
process_folder('input_images', 'output_images')Learning Curve#
Rating: Beginner-friendly (2/5 difficulty)
- Very simple API
- Minimal configuration
- Good for getting started quickly
- Limited customization options
Documentation Quality#
Rating: 7/10
- Good README on GitHub
- Original paper provides theoretical background
- Community examples widely available
- Less comprehensive than MediaPipe/Dlib
- Links:
- Popular implementation: https://github.com/ipazc/mtcnn
- Original paper: https://arxiv.org/abs/1604.02878
7. Pricing/Cost Model#
Free and Open Source
- MIT License (most implementations)
- No usage fees
- Commercial use permitted
- Self-hosted only
8. Integration Ecosystem#
Works With#
- TensorFlow: Most common backend
- PyTorch: Alternative implementations exist
- OpenCV: Image I/O and preprocessing
- NumPy: Landmark manipulation
- Face recognition pipelines: Good as preprocessing step
Output Format#
- Detections: Dictionary with ‘box’, ‘confidence’, ‘keypoints’
- Box: [x, y, width, height]
- Keypoints: 5 points (left_eye, right_eye, nose, mouth_left, mouth_right)
- Confidence: Float (0-1)
- Format: Python dict, easily converted to JSON
Preprocessing Requirements#
- Input: RGB images (convert from BGR if using OpenCV)
- No special preprocessing: Model handles scaling
- Image pyramid: Automatically generated for multi-scale detection
9. Use Case Fit#
Best For#
- General face detection: Balanced accuracy and speed
- Face alignment preprocessing: 5 landmarks useful for alignment
- Legacy systems: Well-established, proven method
- Resource-constrained: Small model size (2 MB)
- Frontal faces: Excellent accuracy on frontal orientations
- Quick prototyping: Simple API, easy setup
Ideal Scenarios#
- Photo organization (face detection for albums)
- Webcam applications (moderate real-time requirements)
- Batch face detection (processing archives)
- Face alignment before recognition
- Research baselines (comparing against MTCNN)
- Edge devices (Raspberry Pi, small model)
Limitations#
- Surpassed by newer methods: RetinaFace, SCRFD more accurate
- Only 5 landmarks: Less detailed than 68-point (Dlib) or 468-point (MediaPipe)
- No face recognition: Detection only
- No face attributes: Age, gender, emotion not provided
- Speed on CPU: Slower than Haar cascades, comparable to Dlib HOG
- Struggles with extreme angles: Best for near-frontal faces
- Less maintained: Original authors not actively updating
10. Comparison Factors#
Accuracy vs Speed#
- Good accuracy: 97.56% AUC (2016 benchmarks)
- Moderate speed: 5-15 FPS on CPU, 20-40 FPS on GPU
- Better than: Haar cascades, early CNNs
- Worse than: RetinaFace, modern YOLO-based detectors
- Sweet spot (2016-2019): Best balance for its era
Self-hosted vs API#
- Self-hosted only: No official cloud API
- Easy deployment: Small model, minimal dependencies
- Privacy-friendly: On-device processing
Landmark Quality#
- 5 points: Basic alignment capability
- Sufficient for: Face alignment, basic geometry
- Less than: Dlib (68), MediaPipe (468), InsightFace (106)
- More than: OpenCV Haar (0), basic detectors
3D Capability#
- No 3D support: 2D landmarks only
Privacy#
- On-device processing: Complete privacy
- No telemetry: No data collection
- GDPR-compliant: Ideal for privacy-sensitive applications
Summary Table#
| Feature | Rating/Value |
|---|---|
| Detection Accuracy | Good (97.56% AUC in 2016) |
| Landmark Count | 5 points (2D) |
| Speed (CPU) | 5-15 FPS |
| Speed (GPU) | 20-40 FPS |
| Model Size | 2 MB (very lightweight) |
| Learning Curve | Beginner-friendly |
| Platform Support | Good (desktop, possible mobile) |
| Cost | Free (MIT License) |
| 3D Support | No |
| Privacy | On-device (excellent) |
When to Choose MTCNN#
Choose MTCNN if you need:
- Lightweight model (2 MB total, perfect for embedded systems)
- Simple API (minimal configuration, quick prototyping)
- 5-point landmarks (basic alignment, less than 68/468 points)
- Legacy compatibility (established method, proven track record)
- Balanced accuracy/speed (for 2016-2019 era standards)
- Face alignment preprocessing (before recognition pipeline)
- Resource constraints (Raspberry Pi, small devices)
Avoid MTCNN if you need:
- State-of-the-art accuracy (use RetinaFace, SCRFD, InsightFace)
- Dense landmarks (use MediaPipe 468-point, Dlib 68-point)
- Face recognition (use InsightFace, Dlib)
- Fastest CPU detection (use Haar cascades, YuNet)
- Production-grade modern solution (use RetinaFace, MediaPipe)
- Face attributes (use commercial APIs)
- Extreme pose handling (use RetinaFace, modern detectors)
Historical Context#
MTCNN was groundbreaking in 2016, offering excellent accuracy and the cascade design was innovative. However, by 2024 standards, it has been surpassed by:
- RetinaFace (2019): Better accuracy, similar speed
- SCRFD (2021): Faster and more accurate
- MediaPipe (2019-2024): Better for mobile/web
- YOLO-based detectors: Faster real-time performance
Still relevant for:
- Legacy systems already using MTCNN
- Educational purposes (understanding cascade detection)
- Resource-constrained devices (small model)
- Quick prototyping (simple API)
Last Updated: January 2025
OpenCV Face Detection Methods#
1. Overview#
What it is: OpenCV (Open Source Computer Vision Library) includes multiple face detection methods, from traditional Haar Cascades to modern DNN-based detectors. A comprehensive computer vision library with face detection as one of many features.
Maintainer: OpenCV Foundation (originally Intel), large open-source community
License: Apache 2.0 (open source, commercial-friendly)
Primary Language: C++ with bindings for Python, Java, JavaScript
Active Development Status:
- Repository: https://github.com/opencv/opencv
- Last updated: Very actively maintained (2024-2025)
- GitHub stars: 78,000+
- Status: Industry standard, mature, production-ready
2. Core Capabilities#
Face Detection Methods in OpenCV#
OpenCV provides three main face detection approaches:
Haar Cascades (2001)
- Traditional, fast, CPU-efficient
- Pre-trained XML classifiers
- Frontal face, profile, eye, smile detection
LBP (Local Binary Patterns) Cascades (2011)
- Faster than Haar, less accurate
- More efficient for real-time embedded systems
DNN Module (2017+)
- Deep learning based (Caffe, TensorFlow models)
- Pre-trained models: ResNet-10 SSD, others
- Higher accuracy than cascades
Facial Landmarks#
- Not included in basic OpenCV
- External libraries needed (Dlib, MediaPipe)
- Can load custom DNN models for landmarks
Face Recognition/Identification#
- Face Recognition module: Built-in algorithms
- Eigenfaces
- Fisherfaces
- LBPH (Local Binary Patterns Histograms)
- Moderate accuracy, educational/simple use cases
- Production systems use Dlib, InsightFace
Face Attributes#
- Not provided
- DNN module can load custom models for attributes
3D Face Reconstruction#
- Not supported
Real-time Performance#
- Haar cascades: 30+ FPS on CPU (very fast)
- DNN module: 15-30 FPS on CPU, 100+ FPS on GPU
3. Technical Architecture#
1. Haar Cascade Classifiers#
How It Works#
- Viola-Jones algorithm (2001)
- Haar-like features (rectangular patterns)
- AdaBoost for feature selection
- Cascade of classifiers (fast rejection)
- Integral images for speed
Pre-trained Models#
- haarcascade_frontalface_default.xml: Standard frontal face (400 KB)
- haarcascade_frontalface_alt.xml: Alternative frontal face
- haarcascade_frontalface_alt2.xml: Another variant
- haarcascade_profileface.xml: Profile faces
- haarcascade_eye.xml: Eye detection
- haarcascade_smile.xml: Smile detection
Custom Training#
opencv_traincascadetool- Requires thousands of positive/negative samples
- Time-consuming (hours to days)
- Limited use in modern era
2. LBP Cascades#
How It Works#
- Local Binary Patterns (texture descriptor)
- Faster than Haar, less accurate
- Good for embedded systems
- Less rotation/lighting invariant
Pre-trained Models#
- lbpcascade_frontalface.xml: LBP frontal face
- lbpcascade_profileface.xml: LBP profile face
3. DNN Module (Deep Neural Networks)#
How It Works#
- Load pre-trained Caffe, TensorFlow, PyTorch, ONNX models
- Forward pass through network
- Post-processing (NMS, thresholding)
Pre-trained Face Detection Models#
ResNet-10 SSD (Caffe)
- 10.4 MB
- Good balance accuracy/speed
- Recommended for most use cases
- Files:
deploy.prototxt,res10_300x300_ssd_iter_140000.caffemodel
OpenCV Face Detector (fp16)
- Optimized 16-bit floating point
- Smaller, faster (5.5 MB)
YOLO Face (community)
- Ultra-fast detection
- Good for real-time applications
Custom Training#
- Load any trained model (Caffe, TensorFlow, PyTorch, ONNX)
- Full flexibility
- Requires external training frameworks
Model Size#
- Haar cascades: ~400 KB - 1 MB each
- LBP cascades: ~200 KB - 500 KB each
- DNN ResNet-10 SSD: 10.4 MB
- DNN fp16: 5.5 MB
Dependencies#
- Core OpenCV: No additional dependencies
- DNN module: Included in OpenCV (3.3+)
- Optional GPU: CUDA support (opencv-contrib)
- Python: NumPy
4. Performance Benchmarks#
Haar Cascades#
Accuracy#
- Frontal faces: Good (70-85% in ideal conditions)
- Profile faces: Poor
- False positives: Common
- Lighting sensitive: Struggles with poor lighting
- Scale sensitive: Multi-scale scanning helps but slow
Speed#
- CPU: 30+ FPS (very fast)
- Real-time: Excellent on any modern CPU
- Embedded: Works on Raspberry Pi
LBP Cascades#
Accuracy#
- Lower than Haar: 60-80% on frontal faces
- Trade-off: Speed over accuracy
Speed#
- Faster than Haar: 40+ FPS on CPU
- Best for: Ultra-low-power devices
DNN Module (ResNet-10 SSD)#
Accuracy#
- Much better than Haar: 85-95% on varied datasets
- Handles angles: Better pose invariance
- Fewer false positives: More robust
- Lighting tolerant: Deep learning handles variations
Speed#
- CPU: 15-30 FPS (640x480 image)
- GPU: 100+ FPS
- Faster than MTCNN, comparable to lightweight RetinaFace
Comparison Table#
| Method | Accuracy | Speed (CPU) | False Positives | Pose Invariance |
|---|---|---|---|---|
| Haar Cascades | Moderate (70-85%) | Very Fast (30+ FPS) | High | Poor |
| LBP Cascades | Lower (60-80%) | Fastest (40+ FPS) | High | Poor |
| DNN ResNet-10 | Good (85-95%) | Fast (15-30 FPS) | Low | Good |
Resource Requirements#
- RAM: 50-200 MB (depending on method)
- GPU memory: 500 MB - 1 GB (DNN with GPU)
- CPU: Efficient, uses all cores
- Disk:
<20MB (all models)
5. Platform Support#
Desktop#
- Windows: ✓ (C++, Python)
- macOS: ✓ (C++, Python)
- Linux: ✓ (C++, Python)
Mobile#
- iOS: ✓ (C++, Objective-C++)
- Android: ✓ (Java, C++ via JNI)
- Official mobile support
Web#
- JavaScript: ✓ (OpenCV.js via WebAssembly)
- Real-time face detection in browser
Edge Devices#
- Raspberry Pi: ✓ (excellent, Haar cascades work well)
- Embedded Linux: ✓
- NVIDIA Jetson: ✓ (DNN module with GPU)
Cloud#
- Easily deployed anywhere
- Docker-friendly
6. API & Usability#
Python API Quality#
Rating: 9/10
- Very clean, intuitive API
- Excellent documentation
- Large community, many tutorials
- cv2.CascadeClassifier, cv2.dnn module
Code Example: Haar Cascade Face Detection#
import cv2
# Load the Haar cascade
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades + 'haarcascade_frontalface_default.xml'
)
# Read image
image = cv2.imread('photo.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Detect faces
faces = face_cascade.detectMultiScale(
gray,
scaleFactor=1.1, # Image pyramid scale reduction
minNeighbors=5, # Minimum neighbors to confirm detection
minSize=(30, 30), # Minimum face size
flags=cv2.CASCADE_SCALE_IMAGE
)
# Draw rectangles
for (x, y, w, h) in faces:
cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)
print(f"Face detected at: ({x}, {y}), size: {w}x{h}")
cv2.imshow('Haar Cascade Detection', image)
cv2.waitKey(0)Code Example: DNN Face Detection (ResNet-10 SSD)#
import cv2
import numpy as np
# Load DNN model
modelFile = "res10_300x300_ssd_iter_140000.caffemodel"
configFile = "deploy.prototxt"
net = cv2.dnn.readNetFromCaffe(configFile, modelFile)
# Optional: Use GPU
# net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
# net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)
# Read image
image = cv2.imread('photo.jpg')
h, w = image.shape[:2]
# Create blob (preprocessing)
blob = cv2.dnn.blobFromImage(
cv2.resize(image, (300, 300)),
1.0,
(300, 300),
(104.0, 177.0, 123.0)
)
# Forward pass
net.setInput(blob)
detections = net.forward()
# Process detections
for i in range(detections.shape[2]):
confidence = detections[0, 0, i, 2]
# Filter by confidence
if confidence > 0.5:
# Get bounding box
box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
(x1, y1, x2, y2) = box.astype("int")
# Draw rectangle
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
# Display confidence
text = f"{confidence * 100:.2f}%"
cv2.putText(image, text, (x1, y1 - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
print(f"Face detected: ({x1}, {y1}), ({x2}, {y2}), confidence: {confidence:.2f}")
cv2.imshow('DNN Face Detection', image)
cv2.waitKey(0)Code Example: Real-time Webcam Detection#
import cv2
# Load DNN model
net = cv2.dnn.readNetFromCaffe('deploy.prototxt', 'res10_300x300_ssd_iter_140000.caffemodel')
# Open webcam
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
h, w = frame.shape[:2]
# Prepare blob
blob = cv2.dnn.blobFromImage(
cv2.resize(frame, (300, 300)),
1.0, (300, 300), (104.0, 177.0, 123.0)
)
# Detect
net.setInput(blob)
detections = net.forward()
# Draw detections
for i in range(detections.shape[2]):
confidence = detections[0, 0, i, 2]
if confidence > 0.5:
box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
(x1, y1, x2, y2) = box.astype("int")
cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
# Display
cv2.imshow('Real-time Face Detection', frame)
# Press 'q' to quit
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Code Example: Haar Cascade with Eye Detection#
import cv2
# Load cascades
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_eye.xml')
image = cv2.imread('photo.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Detect faces
faces = face_cascade.detectMultiScale(gray, 1.1, 5)
for (x, y, w, h) in faces:
# Draw face rectangle
cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)
# Region of interest for eye detection
roi_gray = gray[y:y+h, x:x+w]
roi_color = image[y:y+h, x:x+w]
# Detect eyes within face
eyes = eye_cascade.detectMultiScale(roi_gray)
for (ex, ey, ew, eh) in eyes:
cv2.rectangle(roi_color, (ex, ey), (ex + ew, ey + eh), (255, 0, 0), 2)
cv2.imshow('Face and Eye Detection', image)
cv2.waitKey(0)Learning Curve#
Rating: Beginner-friendly (1/5 difficulty)
- Very easy to get started
- Extensive tutorials everywhere
- Simple API, few parameters
- Most popular CV library worldwide
Documentation Quality#
Rating: 10/10
- Excellent official documentation
- Countless tutorials, courses, books
- Active community (Stack Overflow, forums)
- Links:
7. Pricing/Cost Model#
Free and Open Source
- Apache 2.0 license
- No usage fees
- Commercial use permitted
- No restrictions
- Self-hosted only
8. Integration Ecosystem#
Works With#
- Everything: OpenCV is the standard CV library
- NumPy: Seamless integration
- Matplotlib: Visualization
- TensorFlow, PyTorch: Load models in DNN module
- Dlib, MediaPipe: Often used together
- PIL/Pillow: Image loading
Output Format#
- Haar/LBP: List of rectangles [(x, y, w, h), …]
- DNN: NumPy array of detections [batch, channels, detections, 7]
- Each detection: [?, ?, confidence, x1, y1, x2, y2]
- Format: NumPy arrays, easily serialized
Preprocessing Requirements#
- Grayscale: Haar/LBP cascades require grayscale
- Color: DNN module uses BGR (OpenCV default)
- Resizing: DNN typically resizes to 300x300
- Normalization: DNN handles internally
9. Use Case Fit#
Best For (by Method)#
Haar Cascades#
- Fastest CPU detection: Real-time on any device
- Embedded systems: Raspberry Pi, low-power devices
- Simple frontal face detection: Webcams, basic apps
- Educational: Learning computer vision
- Legacy systems: Already widely deployed
DNN Module#
- Better accuracy: Modern deep learning
- Pose-invariant: Handles varied angles
- Production systems: Reliable, fewer false positives
- GPU acceleration: Fast with GPU
- Flexible: Load any trained model
Ideal Scenarios#
- Video conferencing (blur background based on face)
- Security cameras (detect faces in feed)
- Photo organization (basic face tagging)
- Smart mirrors (detect user presence)
- Attendance systems (detect faces, simple cases)
- Robotics (face tracking)
- Embedded devices (Haar cascades for speed)
- Prototyping (quick face detection setup)
Limitations#
- No landmarks: Need Dlib or MediaPipe for landmarks
- No face recognition: Built-in recognition weak (use Dlib, InsightFace)
- Haar false positives: High false positive rate
- Haar pose limitations: Frontal faces only
- No face attributes: Age, gender, emotion not provided
- DNN accuracy: Good but not state-of-the-art (use RetinaFace for highest accuracy)
10. Comparison Factors#
Accuracy vs Speed#
| Method | Accuracy | Speed | Use Case |
|---|---|---|---|
| Haar Cascades | Moderate (70-85%) | Very Fast (30+ FPS CPU) | Speed critical, frontal faces |
| LBP Cascades | Lower (60-80%) | Fastest (40+ FPS CPU) | Ultra-low-power devices |
| DNN ResNet-10 | Good (85-95%) | Fast (15-30 FPS CPU) | Balance, modern applications |
Self-hosted vs API#
- Self-hosted only: No official cloud API
- Advantage: No costs, privacy, offline
- Universal: Runs everywhere
Landmark Quality#
- None: No landmarks in face detection methods
- Use with: Dlib (68 points), MediaPipe (468 points)
3D Capability#
- No 3D support
Privacy#
- On-device processing: Complete privacy
- No telemetry: No data collection
- GDPR-compliant: Perfect for privacy applications
Summary Table#
| Feature | Haar Cascades | DNN ResNet-10 |
|---|---|---|
| Accuracy | 70-85% | 85-95% |
| Speed (CPU) | 30+ FPS | 15-30 FPS |
| Speed (GPU) | N/A | 100+ FPS |
| Model Size | ~1 MB | 10.4 MB |
| False Positives | High | Low |
| Pose Invariance | Poor | Good |
| Learning Curve | Beginner | Beginner |
| Year Introduced | 2001 | 2017 |
When to Choose OpenCV#
Choose OpenCV Haar Cascades if you need:
- Fastest CPU detection (30+ FPS, any device)
- Embedded systems (Raspberry Pi, low-power)
- Simple frontal face detection (webcams, straightforward scenarios)
- Minimal model size (
<1MB) - Real-time on CPU without GPU
- Legacy compatibility (already deployed everywhere)
Choose OpenCV DNN Module if you need:
- Better accuracy than Haar (85-95% vs 70-85%)
- Modern deep learning detection
- Pose-invariant detection (varied angles)
- Fewer false positives (production quality)
- Flexibility to load any trained model
- GPU acceleration (100+ FPS)
Avoid OpenCV if you need:
- Dense facial landmarks (use MediaPipe 468, Dlib 68)
- State-of-the-art recognition (use InsightFace, Dlib)
- Highest detection accuracy (use RetinaFace, InsightFace)
- Face attributes (age, gender) (use commercial APIs)
- 3D face mesh (use MediaPipe)
Recommendation by Use Case#
| Use Case | Recommended Method | Rationale |
|---|---|---|
| Raspberry Pi project | Haar Cascades | Fast on CPU, minimal resources |
| Webcam app (frontal faces) | Haar Cascades | Real-time, simple |
| Production face detection | DNN ResNet-10 | Better accuracy, robust |
| GPU-accelerated pipeline | DNN ResNet-10 | 100+ FPS with GPU |
| Mobile app (iOS/Android) | DNN ResNet-10 (CoreML/TFLite) | Modern, accurate |
| Learning computer vision | Haar Cascades | Educational, understand basics |
| High-accuracy requirement | External (RetinaFace, InsightFace) | OpenCV good but not SOTA |
Last Updated: January 2025
S1 Verdict: Which Options Merit Deeper Analysis#
S1 research date: January 2025. This file written: 2026-08-25, alongside S2-S4.
Note on This File#
S1 was completed without a recommendation.md; its conclusions are spread
through synthesis.md. This file does the job S1’s recommendation would
have done — naming which options merit an S2 pass and why — using S1’s own
findings, and flags where S2 subsequently disagreed. It is a bridge
between passes, not a revision of S1. The revisions are in
../S2-comprehensive/recommendation.md.
What S1 Established#
Read on its own terms, S1’s durable findings are these:
- No option leads at every job. S1’s own tables show MediaPipe leading on landmark density and mobile support, InsightFace on recognition accuracy, RetinaFace on detection accuracy, and OpenCV on speed and ubiquity. Nothing is best at all of them.
- Model size spans two orders of magnitude — from a ~1 MB Haar cascade to a 200 MB-class recognition model — and on constrained targets that decides the question before accuracy is consulted.
- Licensing is a first-class selection criterion, not a footnote. S1 recorded this as a finding and flagged InsightFace specifically. It did not resolve the chains, and its own licensing table is wrong in three places, but the instinct was right and it is the finding S2 and S4 built on.
- The hosted-versus-self-hosted choice is about where biometric data goes, and cost is the smaller half of it. S1 says this clearly.
- Detection, landmarks and recognition are separable capabilities, and S1’s decision tree branches on them even though its comparison table does not.
Which Options Merited S2#
All eight of S1’s options carried forward, plus four it did not cover.
| Option | Why it merited S2 |
|---|---|
| MediaPipe | The only source of dense 3D landmarks with first-party mobile and web bindings. Its version and landmark count both moved after S1 |
| dlib | The mature all-rounder, and the clearest case of a project whose per-model licenses differ from its package license |
| InsightFace | The accuracy leader and the category’s central licensing question. Shipped a major version and changed its licensing posture after S1 |
| RetinaFace | The most-quoted detection number in the field, and a provenance chain S1 did not follow |
| MTCNN | Widely inherited; needed a maintenance verdict rather than an accuracy one |
| OpenCV | The most-installed option by a factor of fifty, and the one whose capabilities S1 most understated |
| Face++ / Rekognition | The hosted tier, where the question is a counterparty rather than a model |
Added in S2, absent from S1:
| Option | Why it had to be added |
|---|---|
| YuNet | MIT weights, 227 KB, in cv2 since 4.5.4. The best-licensed detector in the category |
| SFace | Apache-2.0 weights, in cv2 since 4.5.4. Completes the only permissively-licensed detection + recognition pair |
| DeepFace | 23,337 stars, actively maintained, and the clearest live demonstration of how model licenses are inherited through a wrapper |
face_recognition | The most-starred project in the category and the most likely thing a reader already depends on. No release since 2020 |
facenet-pytorch | Widely used in research code; dormant since 2024 |
What S2 Was Asked to Settle#
The questions S1 left open, which set S2’s agenda:
- Resolve every license at the weights level, separately from the package, against each project’s own repository and model card. S1 recorded one license per project; several projects have three.
- Attribute every accuracy figure to whoever measured it, and establish which figures share a benchmark and can be compared.
- Check maintenance directly, rather than from stars or a “last updated” impression. S1’s maintenance section lists every project it covers as active.
- Establish what is actually installed — wheels versus source builds, platform coverage, transitive dependencies, and what gets downloaded at runtime.
- Cover the wrapper tier and the OpenCV models S1 omitted.
Where S2 Disagreed With S1#
Summarized here for navigation; the evidence is in the per-project S2 files
and the full list is in ../S2-comprehensive/recommendation.md.
- dlib’s 68-point landmark model carries an author’s note saying it cannot be used in a commercial product. S1’s licensing table records dlib as unrestricted.
- InsightFace’s code is MIT with no commercial limitation; its models are uniformly research-only. S1 recorded the project as a single “mixed”.
- OpenCV has shipped a modern detector and a modern recognizer since 4.5.4. S1 records OpenCV recognition as “weak”.
buffalo_scis the smallest InsightFace pack at 16 MB, not the largest.- MediaPipe’s maintained API produces 478 landmarks, not 468.
pip install dlibcompiles from source; there are no wheels.- A dozen accuracy and speed figures in S1’s tables could not be traced to any source and are recorded as not verified.
Reading Order From Here#
../S2-comprehensive/approach.md— the license chain, and why the category sorts on it../S2-comprehensive/feature-comparison.md— three tables, one per job../S3-need-driven/— six personas, six different answers../S4-strategic/licensing-chain.mdand../S4-strategic/regulatory-surface.md— the two things this category has that most do not
RetinaFace: Single-stage Dense Face Localisation#
1. Overview#
What it is: RetinaFace is a state-of-the-art single-stage face detection framework that performs pixel-wise face localization with multi-task learning. Simultaneously predicts face bounding boxes, 5 facial landmarks, and 3D face information. Known for exceptional accuracy on challenging datasets.
Maintainer: Original paper by Jiankang Deng et al. (Imperial College London, InsightFace team), multiple open-source implementations
License: MIT License (most implementations)
Primary Language: Python (PyTorch, MXNet implementations)
Active Development Status:
- Original paper: 2019 (CVPR)
- Popular implementations:
- https://github.com/serengil/retinaface (3,000+ stars)
- https://github.com/biubug6/Pytorch_Retinaface (7,000+ stars)
- Status: Actively maintained, production-ready, state-of-the-art
2. Core Capabilities#
Face Detection#
- Single-stage detector: No proposal generation (faster than two-stage)
- Multi-scale detection: Feature pyramid network
- High accuracy: State-of-the-art on WIDER FACE benchmark
- Dense predictions: Pixel-wise face localization
- Multi-face detection support
Facial Landmarks#
- 5-point landmarks: Eyes (2), nose (1), mouth corners (2)
- Output simultaneously with detection
- Used for face alignment and quality assessment
Face Recognition/Identification#
- Not included (detection and landmarks only)
- Often used with InsightFace for full pipeline
Face Attributes#
- Not directly provided
- Face quality score from landmark confidence
3D Face Reconstruction#
- 3D face information: Outputs 3D position hints (optional)
- Not full 3D reconstruction
- Helps with pose estimation
Real-time Performance#
- GPU real-time: 30+ FPS on modern GPUs
- CPU acceptable: 5-15 FPS (depending on backbone)
- MobileNet backbone enables mobile deployment
3. Technical Architecture#
Underlying Models#
Single-stage Dense Face Localization#
- Feature Pyramid Network (FPN): Multi-scale feature extraction
- Backbone options:
- ResNet-50/101/152: High accuracy
- MobileNet-0.25: Lightweight, mobile-friendly (1.7 MB)
- VGG-16: Legacy option
- RetinaNet-style: Anchor-based detection with focal loss
Multi-task Learning Branches#
- Classification branch: Face vs. non-face
- Bounding box regression: Face localization
- Landmark regression: 5 facial landmarks
- 3D vertices regression (optional): 3D face position hints
Architecture Details#
- Feature pyramid: 5 levels (P2-P6)
- Context module: Increases receptive field
- SSH modules: Single Stage Headless design
- Deformable convolution: Better geometric variation handling
- Multi-task loss: Weighted sum of all branches
Pre-trained Models#
- ResNet-50 backbone: Best balance (accuracy/speed)
- MobileNet-0.25: Lightweight (1.7 MB, 80.99% WIDER FACE hard)
- ResNet-152: Highest accuracy (91.4% WIDER FACE hard)
- Trained on WIDER FACE dataset
- Available from model zoos (PyTorch, MXNet, ONNX)
Custom Training#
- Fully supported: Training code available
- Datasets: WIDER FACE, custom annotations
- PyTorch training: Most actively maintained
- Configuration: Flexible anchor, loss, augmentation settings
- Documentation: Good training guides
Model Size#
- MobileNet-0.25: 1.7 MB (ultra-lightweight)
- ResNet-50: 30-50 MB
- ResNet-152: 150-200 MB
- Trade-off: Accuracy vs. size/speed
Dependencies#
- PyTorch or MXNet (backend)
- OpenCV: Image processing
- NumPy: Array operations
- torchvision: For PyTorch implementations
- ONNX Runtime: For production deployment (optional)
4. Performance Benchmarks#
Detection Accuracy (WIDER FACE Benchmark)#
Original RetinaFace (ResNet-152)#
- Easy set: 96.3% AP
- Medium set: 95.6% AP
- Hard set: 91.4% AP
- Result: State-of-the-art, 1.1% better than previous best (2019)
Lightweight RetinaFace (MobileNet-0.25)#
- Easy set: 90-94% AP
- Medium set: 88-93% AP
- Hard set: 80-84% AP
- Model size: 1.7 MB only
2024 Performance Reports#
- ResNet-based: 94-96% (easy), 93-95% (medium), 83-91% (hard)
- Improved variants: Up to 94.1% easy, 92.2% medium, 82.1% hard
Comparison with Other Methods (WIDER FACE Hard)#
- RetinaFace (ResNet-152): 91.4%
- MTCNN: 83.55%
- Dlib CNN: Not specifically benchmarked on WIDER FACE
- Haar cascades: ~60-70%
Speed#
| Backbone | Device | Performance |
|---|---|---|
| MobileNet-0.25 | CPU | 10-20 FPS |
| MobileNet-0.25 | GPU | 60+ FPS |
| ResNet-50 | CPU | 3-7 FPS |
| ResNet-50 | GPU | 30-50 FPS |
| ResNet-152 | GPU | 15-25 FPS |
Speed vs. Accuracy Trade-off#
- MobileNet-0.25: Fast, lightweight, 80% hard accuracy
- ResNet-50: Balanced, 85-88% hard accuracy
- ResNet-152: Highest accuracy (91.4%), slower
Latency#
- MobileNet (GPU): 15-30 ms per frame
- ResNet-50 (GPU): 30-60 ms per frame
- CPU latency: 100-300 ms (depending on backbone)
Resource Requirements#
- RAM: 200-500 MB (loaded models)
- GPU memory: 1-3 GB (depending on backbone, batch size)
- CPU: Multi-threaded, moderate to high usage
- Disk: 2 MB - 200 MB (model dependent)
5. Platform Support#
Desktop#
- Windows: ✓ (Python, PyTorch/MXNet)
- macOS: ✓ (Python, PyTorch/MXNet)
- Linux: ✓ (Python, primary platform)
Mobile#
- iOS: ✓ (CoreML conversion, MobileNet backbone)
- Android: ✓ (TFLite conversion, MobileNet backbone)
- MobileNet variant optimized for mobile
Web#
- JavaScript: Possible (ONNX.js, TensorFlow.js)
- Not officially supported
- Community implementations exist
Edge Devices#
- Raspberry Pi: ✓ (MobileNet backbone, acceptable performance)
- Jetson Nano/Xavier: ✓ (excellent with GPU)
- NVIDIA devices: First-class support
- Embedded: ✓ (MobileNet is edge-friendly)
Cloud#
- Easily deployed in cloud (Docker, Kubernetes)
- ONNX export for production
6. API & Usability#
Python API Quality#
Rating: 8/10 (varies by implementation)
- serengil/retinaface: Simple, high-level API (9/10)
- biubug6/Pytorch_Retinaface: Lower-level, more control (7/10)
- Good documentation in popular repos
Code Example: Simple Face Detection (serengil/retinaface)#
from retinaface import RetinaFace
import cv2
# Detect faces (automatically downloads model on first use)
faces = RetinaFace.detect_faces('photo.jpg')
# Read image for visualization
image = cv2.imread('photo.jpg')
# Iterate through detected faces
for key, face in faces.items():
# Bounding box
facial_area = face['facial_area']
x1, y1, x2, y2 = facial_area
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
# Confidence score
score = face['score']
cv2.putText(image, f'{score:.2f}', (x1, y1 - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
# 5-point landmarks
landmarks = face['landmarks']
for landmark_name, point in landmarks.items():
cv2.circle(image, (int(point[0]), int(point[1])), 2, (0, 0, 255), -1)
# Individual landmarks
left_eye = landmarks['left_eye']
right_eye = landmarks['right_eye']
nose = landmarks['nose']
mouth_left = landmarks['mouth_left']
mouth_right = landmarks['mouth_right']
print(f"Face: {key}, Score: {score:.2f}, Box: {facial_area}")
cv2.imshow('RetinaFace Detection', image)
cv2.waitKey(0)Code Example: Custom Model and Threshold#
from retinaface import RetinaFace
# Build model with specific backend
model = RetinaFace.build_model()
# Detect with custom threshold
faces = RetinaFace.detect_faces(
img_path='photo.jpg',
threshold=0.9, # Higher threshold = fewer false positives
model=model,
allow_upscaling=True # Detect smaller faces
)
print(f"Detected {len(faces)} faces with high confidence")Code Example: PyTorch Implementation (biubug6)#
import torch
import cv2
from models.retinaface import RetinaFace
from utils.box_utils import decode, decode_landm
import numpy as np
# Load model
model = RetinaFace(cfg=cfg, phase='test')
model.load_state_dict(torch.load('weights/Resnet50_Final.pth'))
model.eval()
model = model.cuda()
# Prepare image
image = cv2.imread('photo.jpg')
img = np.float32(image)
im_height, im_width, _ = img.shape
# Preprocessing
scale = torch.Tensor([img.shape[1], img.shape[0],
img.shape[1], img.shape[0]])
img -= (104, 117, 123)
img = img.transpose(2, 0, 1)
img = torch.from_numpy(img).unsqueeze(0)
img = img.cuda()
# Forward pass
loc, conf, landms = model(img)
# Post-processing
priorbox = PriorBox(cfg, image_size=(im_height, im_width))
priors = priorbox.forward()
priors = priors.cuda()
boxes = decode(loc.data.squeeze(0), priors.data, cfg['variance'])
boxes = boxes * scale
boxes = boxes.cpu().numpy()
scores = conf.squeeze(0).data.cpu().numpy()[:, 1]
landms = decode_landm(landms.data.squeeze(0), priors.data, cfg['variance'])
# Filter by confidence
inds = np.where(scores > 0.5)[0]
boxes = boxes[inds]
landms = landms[inds]
scores = scores[inds]
# Apply NMS
keep = nms(boxes, scores, nms_threshold=0.4)
boxes = boxes[keep]
landms = landms[keep]
# Draw results
for box, landmark in zip(boxes, landms):
# Bounding box
x1, y1, x2, y2 = map(int, box[:4])
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
# Landmarks (5 points)
landmark = landmark.reshape(-1, 2)
for point in landmark:
cv2.circle(image, tuple(map(int, point)), 2, (0, 0, 255), -1)
cv2.imshow('RetinaFace', image)
cv2.waitKey(0)Code Example: ONNX Runtime Deployment#
import onnxruntime as ort
import cv2
import numpy as np
# Load ONNX model
session = ort.InferenceSession(
'retinaface_resnet50.onnx',
providers=['CUDAExecutionProvider', 'CPUExecutionProvider']
)
# Prepare input
image = cv2.imread('photo.jpg')
input_image = cv2.resize(image, (640, 640))
input_image = input_image.astype(np.float32)
input_image = np.transpose(input_image, (2, 0, 1))
input_image = np.expand_dims(input_image, axis=0)
# Run inference
outputs = session.run(None, {'input': input_image})
# Parse outputs (bounding boxes, scores, landmarks)
boxes, scores, landmarks = outputs[0], outputs[1], outputs[2]
# Apply confidence threshold and NMS
# ... (post-processing logic)Learning Curve#
Rating: Intermediate (3/5 difficulty)
- High-level implementations: Easy (serengil)
- Low-level implementations: Moderate (PyTorch training)
- Good examples available
- Understanding anchors and FPN helps
Documentation Quality#
Rating: 8/10
- Original paper: Excellent theoretical background
- serengil/retinaface: Good README, simple examples
- biubug6/Pytorch_Retinaface: Good for training
- Active community support
- Links:
7. Pricing/Cost Model#
Free and Open Source
- MIT License (most implementations)
- No usage fees
- Commercial use permitted
- Self-hosted only
8. Integration Ecosystem#
Works With#
- PyTorch: Primary framework
- MXNet: Alternative implementation
- ONNX Runtime: Production deployment
- OpenCV: Image I/O and preprocessing
- NumPy: Array operations
- TensorRT: NVIDIA optimization
- CoreML: iOS deployment
- TensorFlow Lite: Android optimization
- InsightFace: Often used together for full face pipeline
Output Format#
- Detections: Bounding boxes [x1, y1, x2, y2], confidence scores
- Landmarks: 5 points (eyes, nose, mouth) as [x, y] coordinates
- Confidence: Float (0-1)
- Format: NumPy arrays or Python dicts (depending on wrapper)
Preprocessing Requirements#
- Input: BGR (OpenCV) or RGB images
- Resolution: Flexible (models handle scaling)
- Normalization: Mean subtraction (104, 117, 123)
- Aspect ratio: Can be maintained or modified
9. Use Case Fit#
Best For#
- High-accuracy requirements: State-of-the-art detection (91.4% WIDER FACE hard)
- Challenging conditions: Occlusions, varied poses, small faces
- Production systems: Robust, well-tested
- Multi-scale detection: Faces of various sizes
- GPU-accelerated pipelines: Real-time with GPU
- Mobile deployment: MobileNet backbone (1.7 MB)
- Research: State-of-the-art baseline
- Face alignment preprocessing: 5 landmarks for recognition pipelines
Ideal Scenarios#
- Security and surveillance (detecting faces in crowds)
- High-resolution image analysis (photos with many faces)
- Challenging lighting conditions (indoor, outdoor, mixed)
- Occluded faces (masks, glasses, hands)
- Wide age range (children to elderly)
- Pose variations (profile, tilted heads)
- Photo organization (accurate face detection for tagging)
- Attendance systems (multiple people in frame)
Limitations#
- Only 5 landmarks: Less detailed than 68-point (Dlib) or 468-point (MediaPipe)
- No face recognition: Detection only, needs separate recognition model
- No face attributes: Age, gender, emotion not provided
- GPU recommended: CPU performance acceptable but slower
- Setup complexity: More complex than high-level libraries (depends on implementation)
- Not 3D mesh: Only 5 landmarks, not full 3D reconstruction
10. Comparison Factors#
Accuracy vs Speed#
- Highest accuracy: 91.4% WIDER FACE hard (ResNet-152)
- Flexible speed: MobileNet (fast) to ResNet-152 (slower)
- GPU-optimized: 30+ FPS with ResNet-50
- Sweet spot: Best accuracy for single-stage detectors
- Better than: MTCNN (83.55%), Dlib, Haar cascades
- Comparable to: SCRFD (newer, similar accuracy, better speed)
Self-hosted vs API#
- Self-hosted only: No official cloud API
- Advantage: No per-call costs, privacy, control
- Easy deployment: ONNX export, Docker-friendly
Landmark Quality#
- 5 points: Basic alignment capability
- Sufficient for: Face alignment before recognition
- Less than: Dlib (68), MediaPipe (468)
- More than: Basic detectors (0)
3D Capability#
- Limited 3D: Optional 3D hints, not full reconstruction
- Use instead: MediaPipe for full 3D mesh
Privacy#
- On-device processing: Complete privacy
- No telemetry: No data collection
- GDPR-compliant: Ideal for privacy-sensitive applications
Summary Table#
| Feature | Rating/Value |
|---|---|
| Detection Accuracy (WIDER FACE Hard) | 91.4% (ResNet-152), 80-84% (MobileNet) |
| Landmark Count | 5 points (2D) |
| Speed (ResNet-50, GPU) | 30-50 FPS |
| Speed (MobileNet, GPU) | 60+ FPS |
| Speed (CPU) | 3-20 FPS (backbone-dependent) |
| Model Size | 1.7 MB (MobileNet) - 200 MB (ResNet-152) |
| Learning Curve | Intermediate |
| Platform Support | Excellent (desktop, mobile) |
| Cost | Free (MIT License) |
| 3D Support | Limited (3D hints only) |
| Privacy | On-device (excellent) |
When to Choose RetinaFace#
Choose RetinaFace if you need:
- State-of-the-art detection accuracy (91.4% WIDER FACE hard)
- Challenging conditions: Occlusions, varied poses, small faces
- Production-grade robustness (well-tested, widely deployed)
- Flexible speed/accuracy trade-off (MobileNet to ResNet-152)
- Mobile deployment (1.7 MB MobileNet model)
- Multi-scale detection (faces of all sizes)
- GPU-accelerated pipeline (real-time 30+ FPS)
- Face alignment preprocessing (5 landmarks before recognition)
Avoid RetinaFace if you need:
- Dense landmarks (68/468 points) for detailed facial analysis (use Dlib, MediaPipe)
- Face recognition (use InsightFace, Dlib)
- Face attributes (age, gender) (use commercial APIs)
- Simplest possible API (use MediaPipe, serengil/retinaface wrapper)
- Full 3D face mesh (use MediaPipe)
- CPU-only fast detection (use Haar cascades, YuNet)
Integration with InsightFace#
RetinaFace is part of the InsightFace ecosystem and is often used as the detection component:
- RetinaFace: Detect faces and 5 landmarks
- Align faces: Use landmarks for alignment
- ArcFace: Extract face embeddings for recognition
This combination provides a complete face detection + recognition pipeline with state-of-the-art performance.
Last Updated: January 2025
Face Detection & Recognition Libraries: Synthesis & Decision Framework#
Executive Summary#
This document synthesizes research on 8 face detection and recognition solutions, providing a comprehensive comparison and decision framework for developers choosing face analysis tools.
Libraries Analyzed:
- MediaPipe Face (Google) - Dense 3D mesh, mobile-optimized
- Dlib - 68-point landmarks, face recognition, mature
- InsightFace - State-of-the-art recognition (99.83% LFW)
- MTCNN - Legacy cascade detector, lightweight
- RetinaFace - Highest detection accuracy (91.4% WIDER FACE hard)
- OpenCV - Traditional (Haar) + modern (DNN) methods
- Face++ API - Commercial, comprehensive attributes
- Amazon Rekognition - AWS cloud service, enterprise-grade
Master Comparison Table#
| Library | Detection Accuracy | Landmarks | Recognition | Speed (CPU) | Model Size | Cost | Best For |
|---|---|---|---|---|---|---|---|
| MediaPipe | 99.3% | 468 (3D) | ✗ | 60-100 FPS | <10 MB | Free | Mobile, AR, 3D mesh |
| Dlib | Good (HOG), Excellent (CNN) | 68 (2D) | 99.38% LFW | 30+ FPS (HOG), 1-3 FPS (CNN) | ~150 MB | Free | Recognition, landmarks |
| InsightFace | 91.4% (WIDER FACE) | 5, 106 (optional) | 99.83% LFW | GPU: 60+ FPS | 50-200 MB | Free (non-commercial) | SOTA recognition |
| MTCNN | 97.56% AUC (2016) | 5 (2D) | ✗ | 5-15 FPS | 2 MB | Free | Lightweight, legacy |
| RetinaFace | 91.4% (WIDER FACE hard) | 5 (2D) | ✗ | GPU: 30-50 FPS | 1.7-200 MB | Free | Highest detection accuracy |
| OpenCV Haar | 70-85% | ✗ | Weak | 30+ FPS | ~1 MB | Free | Fastest CPU, embedded |
| OpenCV DNN | 85-95% | ✗ | Weak | 15-30 FPS | 10 MB | Free | Modern detection, balanced |
| Face++ | 99%+ | 83-106 | ✓ | 200-500 ms API | Cloud | $100+/day | Attributes, cloud |
| AWS Rekognition | High | Basic | ✓ | 100-500 ms API | Cloud | $1/1K images | AWS ecosystem, video |
Accuracy vs Speed Spectrum#
High Accuracy (Detection)
│
├── RetinaFace (ResNet-152): 91.4% WIDER FACE hard [GPU: 15-25 FPS]
├── InsightFace (RetinaFace): 91.4% [GPU: 60+ FPS]
├── MediaPipe: 99.3% comparative [CPU: 60-100 FPS]
├── MTCNN: 97.56% AUC (2016) [CPU: 5-15 FPS, GPU: 20-40 FPS]
├── OpenCV DNN: 85-95% [CPU: 15-30 FPS, GPU: 100+ FPS]
├── Dlib CNN: Excellent [CPU: 1-3 FPS, GPU: 50+ FPS]
├── Dlib HOG: Good (frontal) [CPU: 30+ FPS]
└── OpenCV Haar: 70-85% [CPU: 30+ FPS]
│
Low Accuracy / High SpeedRecognition Accuracy (LFW Benchmark):
- InsightFace (ArcFace): 99.83%
- Dlib: 99.38%
- Face++: 99%+ (proprietary)
- AWS Rekognition: High (exact benchmark not public)
Decision Framework: “Choose X if you need Y”#
By Primary Use Case#
Real-time Video Processing (Webcam, Security Cameras)#
- Simple frontal faces, CPU-only: OpenCV Haar Cascades (30+ FPS)
- Better accuracy, CPU-only: OpenCV DNN ResNet-10 (15-30 FPS)
- GPU available, high accuracy: RetinaFace MobileNet (60+ FPS)
- Mobile device (iOS/Android): MediaPipe (30-60 FPS)
- 3D face tracking: MediaPipe Face Mesh (468 points)
Batch Photo Processing (Photo Libraries, Albums)#
- Face detection only: RetinaFace (highest accuracy, 91.4%)
- Face detection + recognition: InsightFace (99.83% LFW)
- Face detection + 68 landmarks: Dlib
- Simple clustering: Dlib face recognition + DBSCAN
- Cloud processing, attributes: AWS Rekognition (age, gender, emotion)
Mobile AR Applications (Filters, Effects)#
- Dense 3D mesh (468 points): MediaPipe Face Mesh
- Cross-platform (iOS, Android, Web): MediaPipe (official support)
- Lightweight (1.7 MB): RetinaFace MobileNet
- 5-point alignment: MTCNN, InsightFace
Attendance/Access Control Systems#
- Face recognition required: InsightFace (ArcFace, 99.83% LFW)
- CPU-only, moderate accuracy: Dlib (HOG detection + face recognition)
- GPU available: RetinaFace + InsightFace
- Cloud-based: AWS Rekognition, Face++
- Masked faces: InsightFace (masked face models)
Photo Organization/Tagging#
- Self-hosted, high accuracy: Dlib face recognition
- Cloud, comprehensive: AWS Rekognition (search in collections)
- Open-source pipeline: RetinaFace (detection) + InsightFace (recognition)
By Technical Requirement#
Highest Detection Accuracy#
- RetinaFace (ResNet-152): 91.4% WIDER FACE hard
- InsightFace (RetinaFace): 91.4% WIDER FACE hard
- MediaPipe: 99.3% (comparative study)
- Face++: 99%+ (proprietary benchmark)
Highest Recognition Accuracy#
- InsightFace (ArcFace): 99.83% LFW
- Dlib: 99.38% LFW
- Face++: 99%+ (proprietary)
- AWS Rekognition: High (production-grade)
Fastest CPU Performance#
- OpenCV Haar: 30+ FPS (frontal faces only)
- Dlib HOG: 30+ FPS (frontal faces only)
- OpenCV DNN: 15-30 FPS (better accuracy)
- MTCNN: 5-15 FPS (cascade design)
Best Mobile Performance#
- MediaPipe: 30-60 FPS,
<10MB, official mobile SDKs - RetinaFace (MobileNet): 1.7 MB, deployable via CoreML/TFLite
- MTCNN: 2 MB, can be deployed on mobile
Smallest Model Size#
- RetinaFace (MobileNet): 1.7 MB
- MTCNN: 2 MB
- OpenCV Haar: ~1 MB
- MediaPipe:
<10MB
Most Detailed Landmarks#
- MediaPipe: 468 points (3D)
- Face++: 106 points (premium tier)
- Dlib: 68 points (2D)
- InsightFace: 106 points (optional)
- MTCNN, RetinaFace, InsightFace: 5 points
3D Face Mesh Capability#
- MediaPipe: Full 3D mesh (468 vertices with UV coordinates)
- Face++: 3D modeling (advanced tier)
- Others: 2D only
Face Attributes (Age, Gender, Emotion)#
- Face++: Comprehensive (age, gender, 7 emotions, beauty, quality)
- AWS Rekognition: Good (age range, gender, 7 emotions, facial features)
- InsightFace: Limited (age, gender, pose in some models)
- Self-hosted libraries: Require separate models
Privacy-Friendly (On-device Processing)#
- MediaPipe: On-device, no telemetry
- Dlib: On-device, no telemetry
- InsightFace: On-device (ONNX Runtime)
- MTCNN: On-device
- RetinaFace: On-device
- OpenCV: On-device, no telemetry
- Face++, AWS Rekognition: Cloud-based (data sent to servers)
Self-hosted vs Cloud Trade-offs#
Self-hosted Libraries (MediaPipe, Dlib, InsightFace, MTCNN, RetinaFace, OpenCV)#
Advantages:
- Cost: Free (open source), no per-call charges
- Latency:
<50ms (local processing) - Privacy: Data never leaves device (GDPR-compliant)
- Offline: No internet required
- Control: Custom models, fine-tuning
- Scalability: No API rate limits
- Long-term savings: No ongoing costs
Disadvantages:
- Infrastructure: Must deploy and maintain servers/models
- Expertise: Requires ML/CV knowledge
- Updates: Manual model updates
- Limited attributes: Age, gender, emotion require additional models
- DevOps: Deployment, monitoring, scaling
Best for:
- High-volume applications (
>100K faces/month) - Privacy-critical use cases (healthcare, government)
- Real-time requirements (
<50ms latency) - Offline/edge deployments
- Long-term cost optimization
Cloud APIs (Face++, AWS Rekognition)#
Advantages:
- Zero infrastructure: No servers to manage
- Quick start: API calls in minutes
- Comprehensive features: Age, gender, emotion out-of-the-box
- Automatic updates: Models improve automatically
- Scalability: Auto-scaling built-in
- Support: Professional support teams
Disadvantages:
- Cost: $1-10 per 1,000 images (expensive at scale)
- Latency: 100-500 ms per API call
- Privacy: Data sent to third-party servers
- Internet: Requires connectivity
- Vendor lock-in: Proprietary APIs
- Data residency: Compliance challenges (GDPR, regional laws)
Best for:
- Startups/MVPs (quick validation)
- Low-medium volume (
<100K faces/month) - Need comprehensive attributes (age, gender, emotion)
- No ML expertise
- Cloud-first architecture
Platform Support Comparison#
| Platform | MediaPipe | Dlib | InsightFace | MTCNN | RetinaFace | OpenCV | Face++ | AWS |
|---|---|---|---|---|---|---|---|---|
| Windows | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | API | API |
| macOS | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | API | API |
| Linux | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | API | API |
| iOS | ✓ (native) | Limited | ✓ (ONNX) | Possible | ✓ (CoreML) | ✓ | SDK | SDK |
| Android | ✓ (native) | Limited | ✓ (ONNX) | Possible | ✓ (TFLite) | ✓ | SDK | SDK |
| Web (WASM) | ✓ (native) | Experimental | Via ONNX.js | Possible | Via ONNX.js | ✓ (OpenCV.js) | API | API |
| Raspberry Pi | ✓ (9-13 FPS) | ✓ (HOG) | ✓ (lightweight) | ✓ | ✓ (MobileNet) | ✓ (Haar) | API | API |
Generic Use Case Patterns#
1. Security Systems (Surveillance, Access Control)#
Requirements: High accuracy, real-time, face recognition, handle occlusions
Recommended Stack:
- Detection: RetinaFace (91.4% accuracy, handles occlusions)
- Recognition: InsightFace ArcFace (99.83% LFW, masked face support)
- Platform: GPU-accelerated server or Jetson devices
- Alternative: AWS Rekognition (for cloud-based, video streams)
Rationale: Security requires highest accuracy. RetinaFace + InsightFace provides state-of-the-art performance. GPU enables real-time processing of multiple camera feeds.
2. Photo Library Organization (Clustering, Search by Person)#
Requirements: Batch processing, face recognition, scalable
Recommended Stack:
- Small library (
<10K photos): Dlib (simple, mature, 99.38% LFW) - Large library (
>10K photos): InsightFace (99.83% LFW, faster) - Cloud solution: AWS Rekognition (managed, searchable collections)
Rationale: Batch processing allows offline work. InsightFace offers best accuracy for large-scale clustering. AWS Rekognition simplifies infrastructure for cloud deployments.
3. AR Filters/Effects (Snapchat-style, Virtual Try-on)#
Requirements: Dense 3D mesh, real-time mobile, cross-platform
Recommended Solution: MediaPipe Face Mesh
- 468-point 3D mesh
- 30-60 FPS on mobile
- Official iOS, Android, Web support
<10MB model size
Rationale: MediaPipe is purpose-built for AR. Dense mesh enables realistic effects. Cross-platform support reduces development effort.
4. Attendance Tracking (Schools, Offices)#
Requirements: Face recognition, multiple people, cost-effective
Recommended Stack:
- Budget-conscious: Dlib (free, 99.38% LFW, CPU-friendly)
- High accuracy: InsightFace (99.83% LFW, GPU-accelerated)
- Cloud-based: AWS Rekognition (managed, video analysis)
- Masked faces: InsightFace (masked face models)
Rationale: Attendance systems benefit from high accuracy to avoid false positives. Dlib offers great balance for CPU-only systems. InsightFace excels with GPU. AWS simplifies cloud deployments.
5. Age Verification Systems (Online Services, Retail)#
Requirements: Age estimation, real-time, privacy considerations
Recommended Solutions:
- On-device: Custom model on MediaPipe/OpenCV (privacy-friendly)
- Cloud: Face++ or AWS Rekognition (age estimation built-in)
Rationale: Age verification often has privacy requirements. On-device processing with custom age model ensures data privacy. Cloud APIs provide out-of-the-box age estimation but send data to servers.
6. Video Conferencing Effects (Background Blur, Beautification)#
Requirements: Real-time, CPU-friendly, face position detection
Recommended Solutions:
- Simple detection: OpenCV DNN ResNet-10 (15-30 FPS CPU, good accuracy)
- Dense landmarks for effects: MediaPipe (60-100 FPS CPU)
Rationale: Video conferencing needs real-time CPU performance. OpenCV DNN provides good face detection for background segmentation. MediaPipe offers dense landmarks for beautification effects.
7. Customer Analytics (Retail, Events)#
Requirements: Demographics (age, gender), emotion, multiple faces
Recommended Solutions:
- Cloud: Face++ or AWS Rekognition (comprehensive attributes)
- Self-hosted: InsightFace (detection) + custom attribute models
Rationale: Customer analytics benefits from comprehensive attributes. Commercial APIs provide age, gender, emotion out-of-the-box. Self-hosted requires additional attribute models but offers privacy and cost savings at scale.
Migration Paths & Combinations#
Common Pipelines#
Production Recognition Pipeline#
RetinaFace (detection) → InsightFace (recognition)
- Best accuracy combination
- RetinaFace: 91.4% detection
- InsightFace: 99.83% recognitionMobile AR Pipeline#
MediaPipe Face Detection → MediaPipe Face Mesh
- Cross-platform (iOS, Android, Web)
- Real-time 30-60 FPS
- 468-point 3D meshLegacy System Upgrade#
Haar Cascades → OpenCV DNN → RetinaFace
- Progressive improvement
- Minimal code changes (OpenCV API similar)
- Significant accuracy gainsCost Optimization#
AWS Rekognition (MVP) → Self-hosted InsightFace (scale)
- Start with cloud for quick validation
- Migrate to self-hosted when volume increases
- Break-even: ~100K faces/monthPrivacy Implications#
On-device Processing (GDPR-Compliant)#
Libraries: MediaPipe, Dlib, InsightFace, MTCNN, RetinaFace, OpenCV
Privacy Benefits:
- Data never leaves device
- No PII sent to third parties
- Full control over data retention
- Offline operation possible
- GDPR Article 25: Data Protection by Design
Use Cases: Healthcare, government, EU deployments, privacy-conscious consumers
Cloud-based APIs#
Services: Face++, AWS Rekognition
Privacy Considerations:
- Biometric data transmitted to third parties
- Data residency concerns (GDPR Article 44)
- Compliance requirements: SOC 2, HIPAA (AWS), data processing agreements
- Encryption in transit and at rest
- Retention policies vary by provider
Mitigation:
- Encrypt data before transmission
- Use on-premise enterprise SDKs (Face++)
- AWS Panorama for edge processing
- Data processing agreements (DPA)
Licensing Summary#
| Library | License | Commercial Use | Attribution |
|---|---|---|---|
| MediaPipe | Apache 2.0 | ✓ Free | Not required |
| Dlib | Boost | ✓ Free | Not required |
| InsightFace | Mixed | Contact team | Varies by model |
| MTCNN | MIT (implementations) | ✓ Free | Not required |
| RetinaFace | MIT (implementations) | ✓ Free | Not required |
| OpenCV | Apache 2.0 | ✓ Free | Not required |
| Face++ | Commercial | License required | N/A |
| AWS Rekognition | Commercial | Pay-per-use | N/A |
Note: InsightFace requires separate commercial licensing. Check model-specific licenses in model zoo.
Performance Optimization Tips#
CPU Optimization#
- Use lightweight models: MobileNet, Haar cascades
- Reduce resolution: Downscale images before processing
- Skip frames: Process every Nth frame in video
- Multi-threading: Parallelize batch processing
- Choose efficient libraries: OpenCV Haar (30+ FPS) for simple detection
GPU Optimization#
- Batch processing: Process multiple images simultaneously
- Use ONNX Runtime: Efficient inference (InsightFace)
- TensorRT: NVIDIA optimization (RetinaFace, InsightFace)
- Mixed precision: FP16 for faster inference
- Model selection: RetinaFace ResNet-50 (30-50 FPS GPU)
Mobile Optimization#
- Use mobile-first libraries: MediaPipe (official mobile support)
- Quantization: Reduce model size (CoreML, TFLite)
- Lightweight backbones: MobileNet (1.7 MB vs 200 MB)
- On-device acceleration: CoreML (iOS), NNAPI (Android)
- Reduce landmarks: 5-point vs 68-point vs 468-point
Cost Optimization (Cloud APIs)#
- Cache results: Store face embeddings, avoid re-processing
- Batch processing: Group API calls (if supported)
- Hybrid approach: Cloud for attributes, self-hosted for detection
- Threshold monitoring: Detect faces locally, verify with API
- Migration path: Cloud (MVP) → Self-hosted (scale)
Deprecated/Avoid Libraries#
Avoid for New Projects:#
MTCNN (for state-of-the-art needs)
- Why: Surpassed by RetinaFace, SCRFD (2019+)
- When to use: Legacy systems, educational purposes, ultra-lightweight (
<2MB)
OpenCV Haar Cascades (for high accuracy)
- Why: 2001 technology, 70-85% accuracy, high false positives
- When to use: Fastest CPU, embedded systems, frontal faces only
Built-in OpenCV Face Recognition (Eigenfaces, Fisherfaces, LBPH)
- Why: Low accuracy compared to modern methods
- When to use: Educational purposes, extremely simple use cases
Still Relevant:#
- Dlib: Mature, stable, excellent for 68-point landmarks and recognition
- OpenCV DNN: Good balance, widely used, 85-95% accuracy
- MediaPipe: State-of-the-art for mobile, AR, 3D mesh
Future Trends (2025+)#
Emerging Technologies#
- Transformer-based detectors: Replacing CNNs (DETR, YOLOv8+)
- On-device AI acceleration: Apple Neural Engine, Qualcomm AI Engine
- Federated learning: Privacy-preserving face recognition
- 3D face reconstruction: From single image (NeRF, Gaussian Splatting)
- Synthetic data training: Reducing real face dataset requirements
Industry Shifts#
- Privacy regulations: Increased scrutiny on biometric data (GDPR, CCPA, BIPA)
- On-device processing: Shift from cloud to edge (Apple, Google promoting)
- Ethical AI: Bias reduction, fairness in face recognition
- Liveness detection: Combating deepfakes, spoofing attacks
Quick Decision Tree#
START: What is your primary use case?
├─ FACE DETECTION ONLY
│ ├─ Need highest accuracy (91.4%)?
│ │ └─ RetinaFace (ResNet-152)
│ ├─ Need mobile/web support?
│ │ └─ MediaPipe or RetinaFace (MobileNet)
│ ├─ Need fastest CPU (<10 MB, 30+ FPS)?
│ │ └─ OpenCV Haar Cascades
│ └─ Need balance (85-95%, 15-30 FPS)?
│ └─ OpenCV DNN ResNet-10
│
├─ FACE RECOGNITION/IDENTIFICATION
│ ├─ Need state-of-the-art (99.83% LFW)?
│ │ └─ InsightFace (ArcFace)
│ ├─ Need 68-point landmarks + recognition?
│ │ └─ Dlib
│ ├─ Cloud-based, managed service?
│ │ └─ AWS Rekognition or Face++
│ └─ Masked face recognition?
│ └─ InsightFace (masked models)
│
├─ DENSE FACIAL LANDMARKS / 3D MESH
│ ├─ Need 468-point 3D mesh for AR?
│ │ └─ MediaPipe Face Mesh
│ ├─ Need 68-point 2D landmarks?
│ │ └─ Dlib
│ └─ Need 106-point landmarks?
│ └─ InsightFace or Face++
│
├─ FACE ATTRIBUTES (Age, Gender, Emotion)
│ ├─ Cloud-based, comprehensive?
│ │ ├─ Face++ (beauty score, 3D modeling)
│ │ └─ AWS Rekognition (video analysis, celebrity)
│ └─ Self-hosted?
│ └─ InsightFace + custom attribute models
│
└─ CONSTRAINTS
├─ Privacy-critical (GDPR, healthcare)?
│ └─ Self-hosted: MediaPipe, Dlib, InsightFace
├─ Mobile-first (iOS, Android, Web)?
│ └─ MediaPipe (official support)
├─ Cost-sensitive (high volume)?
│ └─ Self-hosted: InsightFace, RetinaFace
├─ Fastest time-to-market (MVP)?
│ └─ Cloud APIs: AWS Rekognition, Face++
└─ Embedded systems (Raspberry Pi)?
└─ OpenCV Haar or MTCNN (lightweight)Recommended Stacks by Developer Profile#
Beginner Developer (Learning CV/ML)#
- Start with: OpenCV Haar Cascades
- Next: OpenCV DNN ResNet-10
- Learn: MediaPipe (modern, good docs)
- Avoid: RetinaFace training, InsightFace setup complexity
Intermediate Developer (Building MVP)#
- Quick prototype: AWS Rekognition or Face++ (cloud)
- Self-hosted: MediaPipe (detection) + Dlib (recognition)
- Mobile: MediaPipe (cross-platform)
- Learn: InsightFace for production
Advanced Developer (Production System)#
- Detection: RetinaFace (highest accuracy)
- Recognition: InsightFace (state-of-the-art)
- Optimize: ONNX Runtime, TensorRT, model quantization
- Scale: Self-hosted for cost efficiency
Startup/Product Team#
- MVP: AWS Rekognition (quick, managed)
- Scale: Migrate to InsightFace when volume increases
- Mobile: MediaPipe (iOS, Android, Web)
- Cost: Break-even analysis at 100K faces/month
Enterprise/Agency#
- Client projects: MediaPipe, OpenCV (permissive licenses)
- Compliance: Self-hosted (GDPR, HIPAA)
- Support: AWS Rekognition (SLA, professional support)
- Custom: InsightFace training pipeline
Key Takeaways#
No one-size-fits-all: Choose based on specific requirements (accuracy, speed, privacy, cost)
Accuracy hierarchy:
- Detection: RetinaFace (91.4%) > InsightFace ≈ MediaPipe (99.3%) > OpenCV DNN (85-95%)
- Recognition: InsightFace (99.83%) > Dlib (99.38%) > Face++ (99%+)
Speed winners:
- CPU: OpenCV Haar (30+ FPS) > Dlib HOG (30+ FPS) > OpenCV DNN (15-30 FPS)
- GPU: OpenCV DNN (100+ FPS) > RetinaFace (30-50 FPS) > InsightFace (30-50 FPS)
- Mobile: MediaPipe (30-60 FPS)
Landmark density:
- MediaPipe: 468 points (3D)
- Face++: 106 points
- Dlib: 68 points
- InsightFace, RetinaFace, MTCNN: 5 points
Privacy-first: MediaPipe, Dlib, InsightFace, OpenCV (on-device processing, no telemetry)
Cost optimization: Self-hosted breaks even at ~100K faces/month vs cloud APIs
Mobile-first: MediaPipe (official support) > RetinaFace MobileNet (1.7 MB)
Production-grade: InsightFace (recognition), RetinaFace (detection), AWS Rekognition (cloud)
Legacy but useful: Dlib (68 landmarks + recognition), OpenCV Haar (fastest CPU)
Avoid for new projects: MTCNN (surpassed), OpenCV Haar (unless speed-critical), built-in OpenCV recognition (low accuracy)
Conclusion#
The face detection and recognition landscape offers diverse solutions for every use case:
- Google’s MediaPipe excels in mobile AR with 468-point 3D mesh
- Dlib remains the gold standard for 68-point landmarks and reliable recognition
- InsightFace delivers state-of-the-art recognition (99.83% LFW) for production systems
- RetinaFace provides highest detection accuracy (91.4% WIDER FACE hard)
- OpenCV offers battle-tested methods from fast Haar to modern DNN
- Face++ and AWS Rekognition simplify cloud deployments with comprehensive attributes
For most developers in 2025:
- Start with: MediaPipe (mobile/web) or OpenCV DNN (server)
- Scale to: InsightFace (recognition) + RetinaFace (detection)
- Optimize: ONNX Runtime, GPU acceleration, model quantization
- Consider cloud: AWS Rekognition for MVPs, migrate to self-hosted at scale
Choose based on your constraints (accuracy, speed, privacy, cost), and combine libraries for optimal results.
Research completed: January 2025 Last updated: January 2025 Version: 1.0
S2: Comprehensive
S2: Comprehensive Analysis — Approach#
Research date: 2026-08-25. Every version number, license term, file size, repository state and download figure in this pass was checked on that date against the source named beside it.
S1 date: January 2025, per its own README. It was written against the state of the field roughly two years earlier than this pass, and several of its verdicts have expired.
What This Pass Asks#
S1 built a comparison table with accuracy on one axis and speed on the other, and produced an ordering. The ordering is not wrong so much as it answers a question almost nobody in this category is actually stuck on.
The question people are stuck on is:
Which of these can I put in the thing I am shipping?
That question is decided by a chain, not by a score:
package → weights → training data
(code) (model) (dataset)A license attaches at each link, and the links do not inherit from each other. A package under MIT can download a model on first run whose terms say research only. A model whose author released it into the public domain can be trained on a dataset whose own terms exclude commercial products — and the model author may say so in his own README, as one in this survey does. The permissive badge on PyPI describes the top link and nothing else.
Resolve the chain for every option in the category and the table reorders. Two of the highest-scoring artifacts in S1’s own accuracy tables turn out to be research-only. A detection-plus-recognition pair that is clear to ship under MIT and Apache-2.0 is already sitting inside the OpenCV install most readers have, and S1 mentions it twice in passing without analysis.
The second thing this pass establishes is that face detection, facial landmarks and face recognition are three separate jobs, and that treating them as one product category is what makes the accuracy tables incoherent. Detecting that a face is present is not the same operation as computing an embedding that identifies a person; the models are different, the benchmarks are different, and — as S4 develops — the legal object is different too. S1’s master table puts a WIDER FACE detection number (91.4%) and an LFW verification number (99.83%) in adjacent columns as if one were larger than the other.
Method#
Primary sources only:
- PyPI JSON API — versions, upload dates, declared license metadata,
dependency lists, and the actual list of distribution files per release.
The file list matters here: it is how this pass established that
pip install dlibstill builds from source. - The GitHub REST API — archive status, license as GitHub resolves it, default branch, star and fork counts, last commit on the default branch, and the open-versus-merged pull request ratio described in ADDING-RESEARCH’s stall test.
- The projects’ own READMEs, LICENSE files and model cards, fetched raw from the default branch.
- Dataset license pages, read at the institution that publishes the dataset — not as quoted by a downstream project.
- Model files themselves, downloaded to measure size, because two of
the sizes quoted in S1 are off by an order of magnitude and the model
files in
opencv_zooare stored in Git LFS, so the repository listing reports the pointer size (131 bytes) rather than the model. - Vendor pricing pages, for the hosted services.
- pypistats.org, for 30-day download counts.
Where a claim could not be checked against one of those, it is marked
not verified or left out. Several of S1’s figures are in that category
and are named in recommendation.md.
Structure of This Pass#
| File | Covers |
|---|---|
mediapipe.md | Google’s BlazeFace + FaceMesh stack; the Tasks/legacy split |
dlib.md | The three dlib face models, which have three different license positions |
insightface.md | ArcFace, the buffalo packs, and the code/weights split |
opencv.md | Haar, the DNN SSD, and the YuNet + SFace pair S1 omits |
retinaface-and-mtcnn.md | The re-implementation layer, and where its weights come from |
wrappers.md | face_recognition, DeepFace, facenet-pytorch: convenience over a chain |
hosted-apis.md | Amazon Rekognition, Face++, Azure Face |
feature-comparison.md | One table per job, rather than one table for all three |
recommendation.md | What the mechanism changes, and what S1 got wrong |
Scope Boundary#
This survey covers libraries and services that detect faces, locate facial landmarks, or produce recognition embeddings. Three neighbors are out of scope and appear only where they change what an option can do:
- Deep learning frameworks (1.075) — PyTorch, TensorFlow, JAX. Several options here are model weights that need one of those, or an ONNX runtime, underneath.
- General image processing (1.080) — decode, resize, color conversion. Every option in this survey assumes you already have a decoded array.
- Vector search (1.009) — once you have an embedding, matching it against a gallery of a million enrolled identities is a nearest-neighbor problem, and the index is a separate choice from the encoder.
Liveness detection, anti-spoofing and face swapping are adjacent product categories that some of these projects also ship. They are noted where a project’s licensing or governance turns on them, and not compared.
Dlib: One Library, Three Models, Three License Positions#
Package: dlib 20.0.1, uploaded 2026-03-29 (PyPI JSON API).
Repository: davisking/dlib — BSL-1.0, 14,431 stars, 3,442 forks,
38 open issues, not archived, last commit 2026-08-11.
Models repository: davisking/dlib-models — CC0-1.0, 1,617 stars,
last commit 2025-05-28.
Downloads: 198,251 in the 30 days to 2026-08-25 (pypistats).
Everything below was checked on 2026-08-25.
The Structural Fact: Dlib Is Not One Face Model#
S1 treats dlib as a single option with a single license — “Boost, ✓ Free, attribution not required”. Dlib’s face capability is three separate artifacts with three separate provenances, and only one of them ships inside the library.
| Artifact | Where it lives | Provenance | Commercial position |
|---|---|---|---|
| HOG frontal face detector | Compiled into the library | Davis King’s own training | Boost Software License, like the rest of dlib |
CNN face detector (mmod_human_face_detector.dat) | dlib-models download | Dataset built by King from ImageNet, AFLW, Pascal VOC, VGG, WIDER, face scrub | Released to public domain by King; the constituent datasets have their own terms |
68-point landmark predictor (shape_predictor_68_face_landmarks.dat) | dlib-models download | iBUG 300-W | Model author states it can’t be used in a commercial product |
5-point landmark predictor (shape_predictor_5_face_landmarks.dat) | dlib-models download | dlib’s own 5-point dataset, 7,198 faces, annotated by King | No dataset restriction stated |
Recognition ResNet (dlib_face_recognition_resnet_model_v1.dat) | dlib-models download | ~3M faces from face scrub, the VGG dataset, and images King scraped | Released to public domain by King; the constituent datasets have their own terms |
That table is the whole finding, and the rest of this file is the evidence for it.
The HOG Detector Ships Inside the Library#
dlib.get_frontal_face_detector() needs no .dat file, and the reason
matters. In dlib/image_processing/frontal_face_detector.h the
detector is base64-encoded into the header itself:
inline frontal_face_detector get_frontal_face_detector()
{
std::istringstream sin(get_serialized_frontal_faces());
frontal_face_detector detector;
deserialize(detector, sin);
return detector;
}The file is 219,037 bytes, nearly all of it that serialized blob, and its second line reads:
// License: Boost Software License See LICENSE.txt for the full license.
The comment further down describes what it is: “It is built out of 5 HOG filters. A front looking, left looking, right looking, front looking but rotated left, and finally a front looking but rotated right one.”
So the HOG detector is the one face model in this survey whose weights carry the same permissive license as the code that runs them, because they are part of that code. Five HOG filters is also why it is a frontal-and-near-frontal detector: there is no filter for a profile.
The 68-Point Predictor Is Research-Only, and dlib Says So#
This is the trap the category is known for, and dlib’s own model
repository documents it. The dlib-models README opens with a blanket
public-domain grant from Davis King — “As far as I am concerned, anyone can
do whatever they want with these model files as I’ve released them into the
public domain” — and then, under shape_predictor_68_face_landmarks.dat,
carries this:
“This is trained on the ibug 300-W dataset … The license for this dataset excludes commercial use and Stefanos Zafeiriou, one of the creators of the dataset, asked me to include a note here saying that the trained model therefore can’t be used in a commercial product. So you should contact a lawyer or talk to Imperial College London to find out if it’s OK for you to use this model in a commercial product.”
The same paragraph appears verbatim under
shape_predictor_68_face_landmarks_GTX.dat, the faster variant from
Alvarez Casado and Bordallo Lopez (2021).
The upstream terms are on Imperial College’s own pages, and they say the same thing in two places. The facial point annotations page:
“All the annotations are provided for research purposes ONLY (NO commercial products).”
and the 300-W challenge page:
“The data are provided for research purposes only. Commercial use (i.e., use in training commercial algorithms) is not allowed.”
A repository-level CC0 grant does not reach past a dataset restriction the model author himself flags in the file’s own documentation. The 68-point landmark model is the single most reproduced artifact in this whole category — it is what almost every “facial landmarks in Python” tutorial loads — and it is not clear to ship.
The 5-point predictor is the way out. Its README entry says it is “trained on the dlib 5-point face landmark dataset … which consists of 7198 faces. I created this dataset by downloading images from the internet and annotating them with dlib’s imglab tool”, with no restriction note. If you need landmarks only for alignment before recognition — which is the common case — five points are what the alignment step wants anyway.
One mechanical caveat from the same README entry: the 68-point model “is designed for use with dlib’s HOG face detector … It won’t work as well when used with a face detector that produces differently aligned boxes, such as the CNN based mmod_human_face_detector.dat face detector.” Swapping the detector under a shape predictor degrades it silently.
The Recognition Model, and Where 99.38% Comes From#
S1 reports “99.38% LFW” for dlib. The number is real, it is the model author’s own, and the README states it with more precision than S1 does:
“The resulting model obtains a mean error of 0.993833 with a standard deviation of 0.00272732 on the LFW benchmark.”
The architecture is “a ResNet network with 29 conv layers … essentially a version of the ResNet-34 network … with a few layers removed and the number of filters per layer reduced by half”, trained from scratch on about 3 million faces across 7,485 identities, drawn from face scrub, the VGG dataset, and “a large number of images I scraped from the internet”. King notes he “made sure to avoid overlap with identities in LFW”, which is the right control and worth crediting.
The loss is described as “a structured metric loss that tries to project
all the identities into non-overlapping balls of radius 0.6” — which is
where the widely copied 0.6 distance threshold in dlib face matching code
comes from. It is a training hyperparameter, not a tuned operating point
for your gallery, and treating it as one is a common source of false
matches at scale.
Two alternates in the same repository are worth knowing about:
taguchi_face_recognition_resnet_model_v1.dat, trained by Taguchi on over
6.5 million faces from over 16,000 people, “Approximately 47% of this is
facial data of Japanese people”, a drop-in replacement for the default
model; and face_recognition_densenet_model_v1.dat from the BAREL project,
at 96.1% LFW with a 0.55 threshold. The existence of the Taguchi model is
itself a data point about the demographic composition of the default one.
pip install dlib Still Compiles From Source#
This is the largest practical fact about dlib in 2026 and S1 does not mention it.
Release 20.0.1 on PyPI contains exactly one file:
dlib-20.0.1.tar.gz sdist 3.3 MBNo wheels. Not for Linux, not for macOS, not for Windows, not for any
Python version. Every pip install dlib runs CMake and a C++ compiler
against 3.3 MB of source. On a machine without a toolchain it fails; on a
CI runner it is minutes of build time on every cold cache; in a Docker
image it means the toolchain is either in the final layer or the build is
multi-stage. On Windows it means Visual C++ Build Tools.
S1 says dlib’s package is “~150 MB”. That number describes a built
installation plus the downloaded .dat files — the 68-point predictor
alone is around 100 MB — not the distribution. The distribution is 3.3 MB
of source and a compiler requirement.
Version history for context: 19.24.9 (2025-05-15), 20.0 (2025-05-28), 20.0.1 (2026-03-29). The major version bump landed in mid-2025; the last release before this pass was five months old, which for a library this mature is a normal cadence rather than a warning.
Governance: Healthy, Small, One Person#
davisking/dlib passes the stall test comfortably:
- Open pull requests: 2
- Pull requests merged since 2026-02-25: 11
- Last commit on
master: 2026-08-11 (“make this test more robust”) - Open issues: 38, against 14,431 stars
Patches go in, the queue is empty, the issue count is tiny for a project of this age and reach. That is the profile of a well-run project.
The concentration risk is the other side of it. Dlib is Davis King’s
project in a way that MediaPipe is not Google’s and OpenCV is not any one
person’s, and dlib-models has not been touched since 2025-05-28. The bus
factor is the thing to weigh, not the activity.
What S1 Got Wrong About Dlib#
- “Boost License, ✓ Free, attribution not required” as a single row for dlib is wrong in the direction that matters. The 68-point landmark model — which S1’s own code samples load, twice — carries a note from its author saying it “can’t be used in a commercial product”.
- "~150 MB package" conflates the install with the distribution. The distribution is a 3.3 MB source tarball and there are no wheels.
- “99.38% LFW” is right but under-attributed. It is the model author’s own measurement of his own model, published in the model’s README, and the full figure is a mean error of 0.993833 ± 0.00272732.
- S1’s platform table gives dlib “Limited” for iOS/Android and “Experimental” for Web. With no wheels at all, the accurate statement is that every platform is a build.
- S1 lists “shape_predictor_68_face_landmarks.dat: 99.7 MB” under model sizes without noting that the file is the restricted one and that a 5-point alternative exists.
Sources#
- dlib on PyPI — version 20.0.1, single sdist, no wheels
- davisking/dlib — repository state;
dlib/image_processing/frontal_face_detector.h - davisking/dlib-models README — per-model provenance and the 300-W restriction note
- iBUG facial point annotations — “research purposes ONLY (NO commercial products)”
- iBUG 300-W — “Commercial use … is not allowed”
- pypistats: dlib
Feature Comparison: Three Jobs, Three Tables#
S1’s master comparison table has one row per project and columns for detection accuracy, landmarks, recognition, speed, model size, cost and “best for”. It is the wrong shape, and it produces a specific error that recurs through S1: a WIDER FACE detection AP and an LFW verification accuracy sit in adjacent columns, inviting the reader to conclude that 99.83 beats 91.4.
They measure different things:
- WIDER FACE asks where are the faces? over 32,203 images with heavy occlusion, scale variation and pose. It scores average precision, split into easy / medium / hard subsets. A “hard” AP of 0.75 is a strong result.
- LFW asks are these two faces the same person? over 6,000 curated pairs, dating from 2007. Modern models score above 99.3 and the benchmark no longer separates them.
One number can be higher than the other without meaning anything. So: three tables, one per job.
All figures checked 2026-08-25. Every accuracy figure is attributed to whoever measured it, because in this category almost all of them are self-reported.
Table 1 — Detection#
Where are the faces?
| Option | Benchmark figure | Measured by | Weights license | Model size |
|---|---|---|---|---|
YuNet (cv2.FaceDetectorYN) | WIDER FACE val: 0.8844 easy / 0.8656 medium / 0.7503 hard | opencv_zoo’s own tools/eval | MIT | 227 KB (98 KB int8) |
| RetinaFace (paper config) | WIDER FACE test: 91.4 AP hard | The paper’s authors (Deng et al., 2019) | Upstream InsightFace: research only | Varies by backbone |
| RetinaFace MobileNet-0.25 | WIDER FACE val: 80.99 hard | biubug6/Pytorch_Retinaface, self-reported | Same upstream question | ~1.7 MB (per S1; not re-verified) |
| MediaPipe BlazeFace short-range | No AP published by Google | — | Apache-2.0 project; model cards published | 128 x 128 input, float16 |
| dlib HOG | None published | — | BSL-1.0 (compiled into the library) | 219 KB header, no download |
dlib CNN (mmod_human_face_detector) | None published | — | Public domain per author | ~700 KB |
| MTCNN | 97.56% AUC (2016), per S1; uncited | Not verified | MIT code; weights bundled | ~2 MB |
| OpenCV Haar | None published | — | Intel License Agreement (BSD-shaped) | ~1 MB |
| OpenCV res10 SSD | None published | — | Apache-2.0 with OpenCV | ~10 MB |
The comparable pair is YuNet at 0.7503 hard (val) against RetinaFace MobileNet at 80.99 hard (val) — same set, same subset, both self-reported by their own projects. RetinaFace MobileNet is ahead by about five points and costs roughly seven times the file size, with an unresolved weights license.
The incomparable pair is YuNet’s 0.7503 (val) against RetinaFace paper’s 91.4 (test). Do not subtract them.
Latency. The only vendor-published latency figure in the category is Google’s: BlazeFace short-range at 2.94 ms CPU / 7.41 ms GPU on a Pixel 6. Every FPS number in S1’s speed table is uncited and is treated here as not verified. Detector latency depends on input resolution, thread count, backend and whether NMS runs on CPU, and a single FPS number without those four is not a measurement.
Table 2 — Landmarks#
Where are the features on this face?
| Option | Points | Dimensionality | Extras | Weights license |
|---|---|---|---|---|
| MediaPipe Face Landmarker | 478 (Tasks API) | 3D | 52 blendshapes, facial transformation matrix | Apache-2.0 project |
MediaPipe legacy face_mesh | 468, or 478 with refine_landmarks=True | 3D | Iris points in the 478 mode | Apache-2.0 project |
| dlib 68-point | 68 | 2D | — | iBUG 300-W: not for commercial products |
| dlib 68-point GTX | 68 | 2D | Faster, smoother | Same 300-W restriction |
| dlib 5-point | 5 | 2D | Built for alignment | No dataset restriction stated |
| InsightFace 2d106 / 3d68 | 106 (2D) and 68 (3D) | Both | Bundled in buffalo packs | Non-commercial research only |
| YuNet | 5 | 2D | Emitted with every detection | MIT |
| MTCNN | 5 | 2D | Emitted by the O-Net stage | MIT code |
| Face++ | up to 106 dense | 2D/3D | 3D reconstruction as a separate product | Commercial |
The finding in this table: the two highest-density landmark options in the category are MediaPipe (478, permissive) and InsightFace (106+68, research only). The 68-point option everyone reaches for by habit is neither the densest nor the most permissive — it is the one with the commercial-use note. If you need five points to align a face before encoding it, YuNet gives them to you for free with the detection you were already doing.
Table 3 — Recognition#
Is this the same person?
| Option | LFW | Harder benchmarks | Measured by | Weights license | Size |
|---|---|---|---|---|---|
| InsightFace buffalo_l | 99.83 | IJB-C(E4) 97.25, CFP-FP 99.33, AgeDB-30 98.23, MR-ALL 91.25 | InsightFace’s model zoo | Non-commercial research only | 326 MB pack |
| InsightFace buffalo_s | 99.70 | IJB-C(E4) 95.02, MR-ALL 71.87 | InsightFace’s model zoo | Same | 159 MB pack |
| InsightFace buffalo_sc | = buffalo_s | = buffalo_s | InsightFace’s model zoo | Same | 16 MB pack |
SFace (cv2.FaceRecognizerSF) | 0.9940 (column labeled “Accuracy”) | None published | opencv_zoo’s tools/eval | Apache-2.0 | 37 MB |
| dlib ResNet | 0.993833 ± 0.00272732 | None published | Davis King, in the model README | Public domain per author; datasets have own terms | ~22 MB |
| dlib Taguchi variant | 0.9895 (as stated) | None published | Taguchi, third party | Public domain per author | ~22 MB |
| dlib BAREL DenseNet | 96.1 at threshold 0.55 | None published | BAREL project | Public domain per author | — |
| MediaPipe | — | — | No recognition model exists | — | — |
| Rekognition / Face++ / Azure | Not published | Not published | — | Commercial | Hosted |
Three readings.
- On LFW, everything ties. 99.83, 99.40, 99.38. Choosing on LFW is choosing on noise. LFW cannot separate these models and has not been able to for years.
- Only InsightFace publishes harder numbers, and they separate sharply: buffalo_l at MR-ALL 91.25 against buffalo_s at 71.87, from models that differ by 0.13 on LFW. If you need to compare recognition models seriously, the benchmark has to be IJB-C or MR-ALL, and for SFace and dlib those numbers do not exist publicly.
- The demographic column exists only for InsightFace, and it shows a spread of ~20 points between best and worst demographic subset on buffalo_l and ~29 points on buffalo_s. The absence of that column for SFace and dlib is not evidence they are even-handed; it is evidence nobody published the measurement.
For third-party numbers on commercial and submitted algorithms, NIST’s FRTE/FATE program is the place to look. NIST describes it as the successor arrangement to FRVT, where “tracks that involve the processing and analysis of images will run under the FATE activity, and tracks that pertain to identity verification will run under FRTE”, and says the program “is an ongoing activity and runs continuously” with participants able to submit roughly every four calendar months. FRTE has 1:1 and 1:N tracks; FATE covers morph detection, quality, presentation attack detection and age estimation. None of the open-source models in this survey are FRTE participants, so no independent number exists for any of them.
Table 4 — What You Are Allowed to Ship#
The table that reorders the other three.
| Stack | Detection | Recognition | Landmarks | All weights clear for commercial use? |
|---|---|---|---|---|
| OpenCV YuNet + SFace | YuNet (MIT) | SFace (Apache-2.0) | 5-point from YuNet | Yes |
| MediaPipe | BlazeFace (Apache-2.0 project) | — none — | 478 3D | Yes, but no recognition |
| dlib, conservative | HOG (BSL-1.0) | ResNet (public domain per author) | 5-point only | Author grants it; dataset provenance disclosed |
| dlib, as commonly used | HOG or CNN | ResNet | 68-point | No — author states the 68-point model can’t be used in a commercial product |
| InsightFace | RetinaFace/SCRFD | ArcFace | 106 + 68 | No — research only; licensing by email |
| retina-face pip package | RetinaFace weights | — | 5-point | Unresolved — MIT code over InsightFace-origin weights |
| DeepFace | your choice | your choice | your choice | Depends on two string arguments |
| Hosted APIs | Yes | Yes | Yes | Contractual, not a license question |
Only one row in that table is an unqualified yes for both jobs, and it is the row S1 does not analyze.
Deployment Surface#
| Option | Desktop Python | Mobile first-party | Web | Install shape |
|---|---|---|---|---|
| OpenCV | Wheels, cp37-abi3, all major platforms | Android/iOS SDKs | OpenCV.js | 34-74 MB wheel |
| MediaPipe | Wheels: macOS arm64 only, manylinux_2_28, Windows | First-party Android, iOS | First-party WASM | 18-38 MB wheel + opencv-contrib-python |
| dlib | No wheels — source build | No | No | 3.3 MB sdist + a C++ toolchain |
| InsightFace | Wheel since 1.0 (2026-05) | Via ONNX Runtime | Via ONNX Runtime Web | Pure-Python wheel + ONNX models |
| retina-face | Wheel | Via TF | — | Wheel + TensorFlow |
| MTCNN | Wheel | Via TF | — | Wheel + TensorFlow |
MediaPipe’s first-party mobile and web bindings are the thing no other option in the survey has, and they are the reason it holds the on-device geometry niche regardless of any accuracy comparison.
Sources#
Every figure in these tables is sourced in the per-project file it comes
from: mediapipe.md, dlib.md, insightface.md, opencv.md,
retinaface-and-mtcnn.md, wrappers.md, hosted-apis.md. The NIST
program description is from
NIST FRTE/FATE.
Hosted APIs: Rekognition, Face++, Azure Face#
The hosted services solve a different problem from the libraries. You are not choosing a model; you are choosing a counterparty — an entity that will hold face images and face templates on your behalf, under terms you did not write, in a jurisdiction you may not have picked.
That framing is the reason this file separates them from the libraries rather than adding two rows to a comparison table.
Everything below was checked on 2026-08-25.
Amazon Rekognition#
What it is. A managed API with two halves. The Image APIs do
detection, attribute analysis, comparison and search; the Video APIs do the
same over stored or streaming video. Face search runs against a
Collection — a server-side store of face vectors you populate with
IndexFaces — so the enrolled gallery lives in AWS, not in your database.
Pricing (aws.amazon.com/rekognition/pricing, US East N. Virginia):
| Volume tier | Group 1 APIs | Group 2 APIs |
|---|---|---|
| First 1 million images | $1.00 per 1,000 | $1.00 per 1,000 |
| Next 4 million | $0.80 per 1,000 | $0.80 per 1,000 |
| Next 30 million | $0.60 per 1,000 | $0.60 per 1,000 |
| Over 65 million | $0.40 per 1,000 | $0.25 per 1,000 |
Face metadata storage is billed separately at “$0.00001/face vectors per month” and “$0.00001/user vectors per month”. The 12-month free tier is 1,000 images per month in each of Group 1 and Group 2, plus 1,000 face vector objects and 1,000 user vector objects per month.
Reading the pricing. At $1.00 per 1,000 images, a service processing 100,000 faces a month pays $100/month; at 10 million a month it pays roughly $7,900 after the tier breaks. The storage line is the one people miss and it is small: a million enrolled faces costs $10/month to keep.
S1’s “$1/1K images” is right for the entry tier. S1’s break-even claim — “self-hosted breaks even at ~100K faces/month” — has no working shown and is not verified; at 100K faces/month the API bill is about $100, which does not obviously beat a GPU instance, and the answer depends entirely on whether the self-hosted option needs dedicated hardware or rides on capacity you already have.
The structural trade. You send images to AWS. That is a data-protection fact before it is an engineering one, and S4 develops it. In return you get no models to license, no weights to update, no ONNX runtime, no GPU, and a face search index that scales without your involvement.
Azure AI Face: The Gated One#
Azure’s face service is in this survey for what it demonstrates rather than for its pricing. Microsoft gates it, and the gate is drawn at exactly the line this survey argues is the important one.
From Microsoft’s own Limited Access documentation:
“Customers and partners who wish to use Limited Access features of the Face API, including Face identification and Face verification, are required to register for access by submitting a registration form. Note that these features are only available with Standard (S0) and Enterprise (E0) pricing tiers and are not supported in the Free (F0) tier. The Face Detection operation is available without registration.”
Emphasis added. Detection is open; identification and verification are gated. That is the detection-versus-identification boundary drawn by a vendor as an access-control policy — the same boundary Texas drew statutorily in 2026 and the same one Annex III of the EU AI Act uses to separate verification from remote identification.
The gate is not a form check:
“Access to Face API is subject to Microsoft’s sole discretion based on eligibility criteria and a vetting process. Face API is available only to customers managed by Microsoft, defined as those customers and partners who are working directly with Microsoft account teams. Additionally, Face API is only available for certain use cases, and customers must select their desired use case in their registration. Microsoft may require customers and partners to reverify this information periodically.”
“Available only to customers managed by Microsoft” means an independent developer with a credit card cannot obtain face identification on Azure.
And a standing prohibition:
“Since the announcement on June 11th, 2020, Azure Face recognition in Foundry Tools is strictly prohibited for use by or for U.S. police departments.”
The page’s own metadata gives ms.date: 2025-10-15 with a last update of
2026-06-05, so this is current policy rather than a historical note.
Why this matters to someone choosing a library. It is the clearest signal in the category that face identification is treated as a different product from face detection by the people who sell both, and that a hyperscaler has concluded the reputational and regulatory exposure justifies turning away revenue. A team weighing self-hosted InsightFace against a hosted API should register that one of the three major clouds will not sell them the hosted version without a relationship and a stated use case.
Face++ (Megvii)#
What it is. The broadest attribute surface in the category. The product list on faceplusplus.com covers Face Detection, Face Comparing (1:1), Face Search (1:N), Face Landmarks, Dense Facial Landmarks, Face Attributes, Emotion Recognition, Beauty Score, Gaze Estimation, Facial Skin Status Analyze, 3D Face Model Reconstruction, and Face Merging, plus body, gesture, OCR and album-clustering products and a set of on-device SDKs. The solutions pages are organized around KYC and identity verification for digital banking, BNPL, P2P lending and crypto.
Pricing. The published price model has four shapes: a Free tier (“Free sign up and try out all Face⁺⁺ services”), Pay As You Go (“Top up account and pay for what you consume”), a Daily/Monthly Plan (“Prepay for future usage by QPS capacity”), and Pay for Licensing (“Billed by license period or license number”). The QPS-capacity model is the distinctive one: you buy concurrency rather than volume, pooled across APIs in the same group. The per-call rates are rendered client-side and this pass could not read them; they are not verified. S1’s “$100+/day” figure has no source and should not be relied on.
The procurement fact. Megvii Technology, the company behind Face++, was added to the U.S. Entity List by the Bureau of Industry and Security effective 2019-10-09. The Federal Register rule (document 2019-22210, “Addition of Certain Entities to the Entity List”) names it in the list of eight:
“The following eight entities are also added to the Entity List as part of this rule: Dahua Technology; Hikvision; IFLYTEK; Megvii Technology; Sense Time, Xiamen Meiya Pico Information Co. Ltd.; Yitu Technologies; and Yixin Science and Technology Co. Ltd.”
The rule’s stated basis for the twenty-eight entities it adds is that they “have been implicated in human rights violations and abuses in the implementation of China’s campaign of repression, mass arbitrary detention, and high-technology surveillance against Uighurs, Kazakhs, and other members of Muslim minority groups” in Xinjiang. The listing imposes “a license requirement for all items subject to the EAR” for exports, reexports and in-country transfers involving the listed entities.
This is reported because it is a fact a reader evaluating vendors needs and will not find in a feature comparison. The Entity List governs exports to a listed entity; whether and how it bears on a particular commercial relationship is a question for counsel, and this survey does not answer it. What is beyond dispute is that many organizations’ procurement processes screen against the list, and that a Face++ dependency will surface there.
The Comparison That Matters#
| Rekognition | Azure Face | Face++ | |
|---|---|---|---|
| Detection available to anyone | Yes | Yes | Yes |
| Identification available to anyone | Yes | No — vetted, managed customers only | Yes |
| Published per-call price | Yes | Yes | Not machine-readable |
| Gallery stored by vendor | Yes (Collections) | Yes (PersonGroups) | Yes (FaceSets) |
| Attribute breadth | Moderate | Moderate | Widest in the category |
| Standing use prohibition | — | US police departments, since 2020-06-11 | — |
| Entity List exposure | — | — | Parent company listed 2019-10-09 |
The generalizable finding is the first two rows. Detection is a commodity that everyone sells to everyone. Identification is not, and one of the three vendors has stopped selling it on open terms. Whatever a reader concludes about any specific vendor, that split is the market’s own statement about which half of this category carries the risk.
The Self-Hosted Comparison, Stated Carefully#
S1’s self-hosted-versus-cloud section is a reasonable list of trade-offs with two claims that do not hold up.
“GDPR-compliant” is not a property of on-device processing. S1 labels the self-hosted libraries “Privacy-Friendly (On-device Processing)” and “GDPR-Compliant”. Processing biometric data for identification on your own hardware is still processing special-category data under GDPR Article 9(1), which prohibits the processing of “biometric data for the purpose of uniquely identifying a natural person” absent one of the Article 9(2) conditions. Local processing removes the international transfer question and the processor relationship. It does not remove the lawful basis question. See S4.
“No telemetry” needs a footnote. It is accurate that MediaPipe, dlib and OpenCV do not phone home during inference. All of the library options except dlib’s built-in HOG detector and OpenCV’s Haar cascade download model weights over the network on first use, from Google Cloud Storage, from GitHub releases, or from InsightFace’s hosts. In an air-gapped or audited deployment those fetches have to be vendored, and the pinned file hash becomes part of your build.
Sources#
- Amazon Rekognition pricing — Group 1/Group 2 tiers, face metadata storage, free tier
- Limited Access features of Face — Microsoft Learn — registration requirement, “managed by Microsoft”, US police prohibition
- faceplusplus.com and its pricing page — product surface and price model shapes
- Federal Register 2019-22210 — Entity List addition, full text read from the Federal Register’s own
full_textendpoint - GDPR Article 9 — special categories, biometric data for unique identification
InsightFace: MIT Code, Research-Only Weights#
Package: insightface 1.0.1, uploaded 2026-05-23 (PyPI JSON API).
Repository: deepinsight/insightface — 29,569 stars, 6,076 forks,
1,266 open issues, not archived, last commit 2026-07-27.
Repository license as GitHub resolves it: null. There is no
LICENSE file at the repository root; a request for one returns 404.
PyPI declared license: none. The 1.0.1 changelog says why.
Downloads: 1,081,113 in the 30 days to 2026-08-25 (pypistats).
Everything below was checked on 2026-08-25.
The License, Stated Three Times by the Project#
InsightFace is the reason “check the weights separately” is the first rule of this category. The project is not ambiguous about it — it states the split in three separate documents, in almost the same words.
The repository README:
“The code of InsightFace is released under the MIT License. There is no limitation for both academic and commercial usage.
>The training data containing the annotation (and the models trained with these data) are available for non-commercial research purposes only.>Both manual-downloading models from our github repo and auto-downloading models with our [python-library] follow the above license policy(which is for non-commercial research purposes only).”
The model zoo README, as its first line after the title:
“:bell: ALL models are available for non-commercial research purposes only.”
The Python package README:
“The code of InsightFace Python Library is released under the MIT License. There is no limitation for both academic and commercial usage.
>The pretrained models we provided with this library are available for non-commercial research purposes only, including both auto-downloading models and manual-downloading models.”
The clause that closes the loophole is the auto-download one. The canonical InsightFace snippet is three lines:
from insightface.app import FaceAnalysis
app = FaceAnalysis(name='buffalo_l')
app.prepare(ctx_id=0)FaceAnalysis downloads buffalo_l on first call. Nobody agreed to
anything, nobody clicked through a model card, and the weights that landed
in ~/.insightface/models/ are research-only by the project’s own
statement. That is the standard trap in this category, and it is not
hidden — it is in the README of the package you installed.
As of 2025-11-24 the README adds a licensing contact per model family:
“1. For inswapper series face swap models (e.g., inswapper_128.onnx/inswapper-512-live), please contact [email protected] for licensing and additional support. 2. For open-sourced face recognition models (e.g., buffalo_l package), please contact [email protected] for licensing. 3. For advanced face recognition SDK and models (e.g., InspireFace SDK), please contact [email protected] for licensing and additional support.”
So a commercial license exists and is transacted by email. That is a different position from “you cannot use this” and a different position again from “it is free”. A reader who wants InsightFace accuracy in a product has a path; it runs through a negotiation, not through pip.
The PyPI metadata now says nothing at all. The 1.0.1 changelog entry, dated 2026-05-23, reads:
“Remove the PyPI package metadata license classifier field while keeping the README license guidance.”
Which is why insightface on PyPI shows no license and no license
classifier, and why any automated license scanner run over a dependency
tree containing InsightFace reports unknown rather than MIT — and
would have reported MIT before May 2026, which was the more misleading
answer.
What the Models Actually Are#
The model zoo’s Python-package table (reproduced here as published):
| Name | Detection Model | Recognition Model | Alignment | Attributes | Model-Size |
|---|---|---|---|---|---|
| antelopev2 | RetinaFace-10GF | ResNet100@Glint360K | 2d106 & 3d68 | Gender&Age | 407MB |
| buffalo_l | RetinaFace-10GF | ResNet50@WebFace600K | 2d106 & 3d68 | Gender&Age | 326MB |
| buffalo_m | RetinaFace-2.5GF | ResNet50@WebFace600K | 2d106 & 3d68 | Gender&Age | 313MB |
| buffalo_s | RetinaFace-500MF | MBF@WebFace600K | 2d106 & 3d68 | Gender&Age | 159MB |
| buffalo_sc | RetinaFace-500MF | MBF@WebFace600K | - | - | 16MB |
buffalo_l is the default. It is a pack, not a model: a detector, a
recognizer, a 106-point 2D and 68-point 3D alignment model, and gender/age
attribute heads, shipped together at 326 MB.
buffalo_sc is the interesting entry for constrained targets — 16 MB,
detection plus recognition, with alignment and attributes stripped out.
The Accuracy Table, Including the Part Everyone Quotes and the Part Nobody Does#
The model zoo publishes this for the two packs it benchmarks:
| Name | MR-ALL | African | Caucasian | South Asian | East Asian | LFW | CFP-FP | AgeDB-30 | IJB-C(E4) |
|---|---|---|---|---|---|---|---|---|---|
| buffalo_l | 91.25 | 90.29 | 94.70 | 93.16 | 74.96 | 99.83 | 99.33 | 98.23 | 97.25 |
| buffalo_s | 71.87 | 69.45 | 80.45 | 73.39 | 51.03 | 99.70 | 98.00 | 96.58 | 95.02 |
with the notes “buffalo_m has the same accuracy with buffalo_l” and “buffalo_sc has the same accuracy with buffalo_s”.
99.83 LFW is the number S1 quotes. It is in the rightmost half of that table for a reason: LFW is a 2007 verification benchmark that modern recognition models saturate, and the spread between buffalo_l (99.83) and buffalo_s (99.70) is thirteen hundredths of a point. On the harder columns the same two models are 91.25 versus 71.87. Anyone choosing between them on LFW is reading the column that cannot distinguish them.
The demographic columns are the ones that should change a decision. buffalo_l scores 94.70 on Caucasian and 74.96 on East Asian — a spread of almost twenty points, published by the model’s own authors. On buffalo_s the spread is 80.45 against 51.03. This is not a third-party audit or an adversarial finding; it is the vendor’s own table, and it is the single most decision-relevant thing on the page for anyone deploying identity matching against a population that is not uniformly represented in WebFace600K. S1 does not mention it.
The training sets themselves are named in the table and matter for the license chain: WebFace600K for the buffalo family, Glint360K for antelopev2, and elsewhere in the zoo MS1MV2 and MS1MV3, which the zoo’s own definitions section identifies as MS1M-ArcFace and MS1M-RetinaFace.
The 1.0 Release Changed the Package Materially#
InsightFace sat at 0.7.3 from 2023-04-02 until 1.0 and 1.0.1 both shipped on 2026-05-23. That is three years and one month of no releases followed by a major version, and it changes three practical things.
Wheels. 0.7.3 was an sdist only, which is why “pip install insightface
needs Visual C++ Build Tools” is a permanent fixture of the project’s issue
tracker. 1.0 ships insightface-1.0-py3-none-any.whl. The Cython
dependency moved to an optional face3d extra.
Dependencies. The base install is now numpy, onnx, onnxruntime,
opencv-python, tqdm, requests, scipy, scikit-image. Pillow,
scikit-learn, reportlab and PySide6-Essentials>=6.5 moved to a gui
extra; matplotlib moved to face3d. A leaner base install than 0.7.3.
A desktop GUI arrived. The 1.0 changelog adds “InsightFace Evaluation
Studio, a cross-platform PySide6 desktop GUI for local face recognition,
album grouping, enterprise evaluation/report export, and face swap trials”,
with a companion document, docs/commercial_evaluation.md, whose opening
paragraph is unusually direct about what the GUI is for:
“Enterprise Evaluation Mode helps teams evaluate InsightFace locally with their own data before choosing a commercial model license, private model evaluation, SDK/API access, SLA, or custom training path.”
Read together with the 2025-11-24 licensing contacts, the shape of the
project in 2026 is clear: an open research codebase functioning as the top
of a commercial funnel. The evaluation tooling is good and its output is
useful — it reports “the best cosine threshold accuracy, the selected
threshold, positive and negative pair counts, and TAR@FAR for 1e-6,
1e-5, 1e-4, and 1e-3 with corresponding thresholds”, which is the
right way to characterize a matcher and better than any LFW figure.
And a server. The README’s top news entry, dated 2026-07-27, announces InsightFace Server as “a simple AWS Rekognition alternative with accuracy-preserving INT8 embedding quantization and 50M+ image search on one RTX 5090 GPU”. That is the project moving into the hosted-API competitors’ territory, under the same model license.
Governance and the Stall Test#
- Open pull requests: 43
- Pull requests merged since 2026-02-25: 3
- Last commit on
master: 2026-07-27 (“build(server): update packaging dependencies”) - Open issues: 1,266
That issue count against 29,569 stars is the highest ratio in this survey by a wide margin. The merge ratio is thin but not zero, and the commits are recent. The reading is a project with an active core moving in a commercial direction and a large community backlog it is not servicing. For an adopter that means: the models will keep getting better, and your bug report will sit.
The missing LICENSE file is a governance detail with practical
consequences. GitHub reports license: null; a dependency scanner reports
unknown; the actual terms are prose in three READMEs. The terms are clear,
but they are not machine-readable anywhere, which is how they get lost in
a procurement review.
RetinaFace Lives Here#
A framing correction that S1’s structure obscures: RetinaFace is an
InsightFace project. The RetinaFace paper (arXiv:1905.00641) lists its
code at github.com/deepinsight/insightface/tree/master/RetinaFace, and
the buffalo packs use RetinaFace variants as their detector. S1 presents
“RetinaFace” and “InsightFace” as two competing entries in the same table,
with a footnote that InsightFace “uses RetinaFace”. They are one lineage
from one lab, and the license position that applies to InsightFace weights
applies to the original RetinaFace weights for the same reason. See
retinaface-and-mtcnn.md.
What S1 Got Wrong About InsightFace#
- “License: Mixed / Contact team / Varies by model” understates it. The models are uniformly research-only by the project’s own statement, including the ones the library downloads for you without asking.
- “Free (non-commercial)” in the cost column is right about the models and wrong about the code, which is MIT with no commercial limitation. The distinction is the whole point of the category.
- “buffalo_sc: High accuracy, larger model” is backwards. The model
zoo lists
buffalo_scat 16 MB — the smallest pack in the family, with alignment and attributes removed, and withbuffalo_s-level accuracy (MR-ALL 71.87) rather thanbuffalo_l-level (91.25). - “buffalo_l model: 99.88% detection success on LFW” does not correspond to any figure in the model zoo. Not verified.
- “Model size 50-200 MB” is low. The published pack sizes are 16 MB
(
buffalo_sc), 159 MB (buffalo_s), 313 MB (buffalo_m), 326 MB (buffalo_l) and 407 MB (antelopev2). - S1’s “Actively maintained (2024-2025)” was true of the repository and missed that the package had not shipped a release since April 2023. That changed in May 2026.
- No mention of the published demographic performance spread, which is on the same table as the LFW figure S1 does quote.
Sources#
- insightface on PyPI — versions, files, dependencies, absent license metadata
- deepinsight/insightface README — MIT code / non-commercial models, 2025-11-24 licensing contacts, 2026-07-27 server announcement
- InsightFace Model Zoo — pack composition, sizes, accuracy and demographic tables
- InsightFace Python Library README — auto-download license clause, 1.0/1.0.1 changelog
- InsightFace commercial evaluation doc
- RetinaFace paper, arXiv:1905.00641
- pypistats: insightface
MediaPipe: Detection and Landmarks, No Recognition#
Package: mediapipe 1.0.1, uploaded 2026-08-14 (PyPI JSON API).
Repository: google-ai-edge/mediapipe — Apache-2.0, 36,723 stars,
6,135 forks, not archived, last commit on master 2026-08-25.
Declared license: Apache 2.0, with the OSI Apache classifier.
Downloads: 3,022,607 in the 30 days to 2026-08-25 (pypistats).
Everything below was checked on 2026-08-25.
What It Is, Mechanically#
MediaPipe is a C++ graph runtime with language bindings, plus a set of
pre-built graphs Google calls Tasks. A Task is a .task file — a
bundle containing one or more TFLite/LiteRT models plus the graph wiring
that connects them — and a thin API per platform that loads it.
Two of those Tasks are in this category:
Face Detector. Runs a BlazeFace model and returns bounding boxes plus a few key landmarks. Google’s documentation lists three variants, all float16 with a 128 x 128 input shape: BlazeFace short-range (front camera, selfie distance), BlazeFace full-range (rear camera), and BlazeFace Sparse full-range, which the docs describe as “roughly 60% smaller in size” than the regular full-range model. The architecture note on the page is that short-range “uses a Single Shot Detector (SSD) convolutional network technique with a custom encoder” while full-range “uses a technique similar to a CenterNet convolutional network”.
Face Landmarker. Chains three models in one bundle. The documentation states the sequence directly:
“The first model detects faces, a second model locates landmarks on the detected faces, and a third model uses those landmarks to identify facial features and expressions.”
with input shapes given as FaceDetector 192 x 192, FaceMesh-V2 256 x 256, and Blendshape 1 x 146 x 2, all float16. The mesh model “outputs an estimate of 478 3-dimensional face landmarks” and the blendshape model “predicts 52 blendshape scores, which are coefficients representing facial different expressions” [sic]. The Task also optionally emits a facial transformation matrix, which the docs describe as what “FaceLandmarker uses … to transform the face landmarks from a canonical face model to the detected face, so users can apply effects on the detected landmarks.”
There is no recognition model. MediaPipe has no face embedding Task, no identity matching, no enrollment gallery. It produces geometry. This is the single most important structural fact about it and it is a design position rather than a gap — Google ships a generic Image Embedder Task, and it is not a face recognition model.
The 468 / 478 Question#
S1 states “468-point 3D face mesh” throughout, and both numbers are real and describe different code paths.
The legacy Solutions API — mediapipe.python.solutions.face_mesh,
still present in master under an Apache-2.0 header dated 2020 — returns
468 landmarks by default and 478 when constructed with
refine_landmarks=True, the extra ten being iris points.
The Tasks API — mediapipe.tasks.python.vision.FaceLandmarker — is
what the current documentation describes, and the current documentation
says 478.
Any reader following S1 to a number will find a different one in the docs Google is maintaining. Both APIs ship in the same wheel today, which is why nothing breaks and nothing warns.
Performance: What Google Publishes and What It Covers#
The Face Detector page carries one benchmark table, and it is narrow:
| Model | CPU latency | GPU latency |
|---|---|---|
| BlazeFace (short-range) | 2.94 ms | 7.41 ms |
with the note “The latency result is the average latency on Pixel 6 using CPU / GPU.”
Three things follow. First, this is a vendor benchmark on one device and should be attributed that way. Second, GPU is slower than CPU here, which is what you would expect for a 128 x 128 model where the upload and download dominate — the reader planning to “add a GPU” for MediaPipe face detection should read that row twice. Third, Google publishes no CPU/GPU latency table for the Face Landmarker bundle at all, so S1’s “60-100 FPS” for MediaPipe has no source on Google’s pages; treat it as not verified.
The useful engineering shape is that 2.94 ms of detector on a 2021 phone
leaves a large frame budget, and that the mesh and blendshape models run
per detected face, so cost scales with face count and num_faces (default
- is the knob.
Packaging and Platform Reach#
mediapipe 1.0.1 ships five wheels and no others:
mediapipe-1.0.1-py3-none-macosx_11_0_arm64.whl 33.7 MB
mediapipe-1.0.1-py3-none-manylinux_2_28_aarch64.whl 35.8 MB
mediapipe-1.0.1-py3-none-manylinux_2_28_x86_64.whl 37.9 MB
mediapipe-1.0.1-py3-none-win_amd64.whl 20.1 MB
mediapipe-1.0.1-py3-none-win_arm64.whl 17.7 MBThere is no macOS x86_64 wheel and no sdist. An Intel Mac cannot
pip install mediapipe at 1.0.1. The manylinux_2_28 floor rules out
older glibc distributions. The wheels carry a py3-none Python tag rather
than a cp3xx ABI tag, so pip will install them under any Python 3; the
package’s own classifiers list 3.9 through 3.12, and the distribution
declares no Requires-Python. That combination means an unsupported
Python gets a successful install and a runtime failure rather than a
resolver error.
Python is the smallest surface here. The reason MediaPipe dominates on-device face geometry is that the same Task bundles run through first-party Android, iOS and Web (WASM) APIs, which is a claim no other option in this survey can make.
Runtime dependencies are light and one of them is notable:
absl-py~=2.3, certifi, numpy, sounddevice~=0.5, flatbuffers~=25.9,
opencv-contrib-python, matplotlib. Installing MediaPipe installs a
second, larger computer vision library as a hard dependency, and
opencv-contrib-python conflicts with opencv-python if something else in
the environment asked for that instead.
Model Licensing#
The code is Apache-2.0 and GitHub resolves the repository license to
Apache-2.0. The .task bundles are distributed by Google from
storage.googleapis.com and each model in the Face Detector and Face
Landmarker pages links to a Model Card. The documentation pages
themselves carry the standard Google Developers footer: content under
CC BY 4.0, code samples under Apache 2.0.
MediaPipe is the one option in this survey where no license reading turned up a research-only restriction on the shipped weights. That is the reason it appears in so many products, and it outweighs any of its accuracy numbers.
Governance and the Pull Request Signal#
Applying ADDING-RESEARCH’s stall test to google-ai-edge/mediapipe returns
a number that looks alarming and is not:
- Open pull requests: 199
- Pull requests merged since 2026-02-25: 0
Read alone, that is the Ragas pattern. It is not, and the reason is
visible in the commit log: MediaPipe is developed inside Google and
synced out. The last commit on master was made on the day this pass ran
(“Remove unistd.h header import”), and the v1.0.0 release notes list dozens
of internal changes — Abseil bumps, WebGPU work, LiteRT accelerator
documentation, EXIF orientation support in the Python Task API. Community
pull requests accumulate because they are not the input path, not because
nobody is working.
The stall test needs a companion check for copybara-style projects: compare open PRs against commits, not against merges. Here the commits are daily and the merges are zero, which is a governance shape rather than a health problem. What it does tell an adopter is real, though — an outside patch is unlikely to land, so a MediaPipe bug you need fixed is a bug you wait for.
The repository moved from google/mediapipe to google-ai-edge/mediapipe;
the old path still redirects, and both resolve to the same repository via
the API. S1’s links to google/mediapipe still work.
The version jump is the other governance signal. MediaPipe sat on 0.10.x
for years and shipped 1.0.0 on 2026-07-27, then 1.0.1 on 2026-08-14.
S1 predates that entirely.
What S1 Got Wrong About MediaPipe#
- “468 points” is the legacy Solutions default. The maintained Tasks API and the current documentation say 478.
- “60-100 FPS” on CPU has no source on Google’s pages. The only benchmark Google publishes for face detection is 2.94 ms CPU / 7.41 ms GPU for BlazeFace short-range on a Pixel 6.
- “99.3% (comparative study)” for MediaPipe detection accuracy is
placed in S1’s master table alongside RetinaFace’s 91.4% WIDER FACE hard,
inviting the reading that MediaPipe detects better than RetinaFace. The
study is not named and the figure is not a WIDER FACE number. Treat it
as not verified, and see
feature-comparison.mdfor why the two cannot share a column. - "
<10MB" describes the model bundle, not the install. The wheel is 20-38 MB and pulls inopencv-contrib-python. - S1’s platform table marks MediaPipe “✓” for macOS without qualification. At 1.0.1 there is an arm64 wheel and no x86_64 wheel.
Sources#
- mediapipe on PyPI — version, files, dependencies, classifiers
- google-ai-edge/mediapipe — repository state, releases,
mediapipe/python/solutions/face_mesh.py - Face landmark detection guide — 478 landmarks, 52 blendshapes, model bundle input shapes (page footer: “Last updated 2026-08-17 UTC”)
- Face detection guide — BlazeFace variants, Pixel 6 latency table
- pypistats: mediapipe
OpenCV: Four Face Models, and the Two S1 Missed#
Package: opencv-python 5.0.0.93 (uploaded 2026-07-02) and
4.14.0.94 (uploaded 2026-07-29) — two live release lines.
Repository: opencv/opencv — Apache-2.0, 90,601 stars, 56,972 forks,
default branch 5.x, last commit 2026-08-25.
Model zoo: opencv/opencv_zoo — Apache-2.0, 1,033 stars, last commit
2026-05-28.
Downloads: 53,780,796 in the 30 days to 2026-08-25 (pypistats) —
roughly 50x the next library in this survey.
Everything below was checked on 2026-08-25.
The Correction That Matters Most in This Survey#
S1’s verdict on OpenCV is “Haar cascades (fast, 70-85%), OpenCV DNN (85-95%), recognition weak”. That was accurate about OpenCV in 2018. Since 4.5.4 OpenCV has shipped two additional face modules in the core library, and they are the answer to the licensing problem the rest of this category has:
| API | Job | Model | License | Size |
|---|---|---|---|---|
cv2.FaceDetectorYN | Detection + 5 landmarks | YuNet | MIT | 227 KB |
cv2.FaceRecognizerSF | 128-d recognition embedding | SFace | Apache-2.0 | 37 MB |
Detection and recognition, both permissively licensed at the weights level,
both callable from the opencv-python most readers already have installed,
totaling under 38 MB. S1 names YuNet twice — once in retinaface.md and
once in mtcnn.md, both times in a one-line “use Haar cascades, YuNet”
aside — and never analyzes it. SFace does not appear in S1 at all.
YuNet#
What it is. A tiny anchor-free face detector from Shiqi Yu’s group at
SUSTech, published as Wu, Peng and Yu, “Yunet: A tiny millisecond-level
face detector”, Machine Intelligence Research 20(5):656-665, 2023. The
ONNX distributed by opencv_zoo is exported from
ShiqiYu/libfacedetection.train.
Accuracy. Two figures on the same model card, from two evaluations, and the difference matters:
The card’s headline text: “achieves 0.834(AP_easy), 0.824(AP_medium), 0.708(AP_hard) on the WIDER Face validation set.”
The card’s own evaluation table, produced by opencv_zoo/tools/eval:
| Model | Easy AP | Medium AP | Hard AP |
|---|---|---|---|
| YuNet | 0.8844 | 0.8656 | 0.7503 |
| YuNet block-quantized | 0.8845 | 0.8652 | 0.7504 |
| YuNet int8 quantized | 0.8810 | 0.8629 | 0.7503 |
Both are WIDER FACE validation-set numbers. The 0.7503 hard AP is the
one to compare against RetinaFace’s 91.4 — with the caveat, developed in
feature-comparison.md, that RetinaFace’s figure is on the test set and
from the paper’s authors.
The quantization result is the striking one: int8 quantization costs 0.0034 AP on easy, 0.0027 on medium, and nothing measurable on hard, while halving the file to 98 KB. That is an unusually clean quantization story and it is why YuNet turns up on microcontrollers.
Known limit, stated on the card: “This model can detect faces of pixels between around 10x10 to 300x300 due to the training scheme.” A face filling a 1080p frame is outside the model’s trained range. Downscale, or use a different detector.
OpenCV 5 caveat, also on the card: the default is now
face_detection_yunet_2026may.onnx, re-exported with symbolic height and
width dims “compatible with OpenCV 5.x ONNX Runtime engine
(OPENCV_FORCE_DNN_ENGINE=4)”. The older 2023mar file has fixed input
shape: “OpenCV 4.x DNN infers on the exact shape of input image, but the
ONNX Runtime engine in OpenCV 5.x requires dynamic dims for variable input
sizes.” Code that pins the 2023mar file and moves to OpenCV 5 will need to
resize or switch files.
License. models/face_detection_yunet/LICENSE is the MIT License,
“Copyright (c) 2020 Shiqi Yu”. The upstream ShiqiYu/libfacedetection
repository carries a 3-clause BSD license (GitHub reports the repository
license as NOASSERTION because the file is a custom-formatted BSD text);
libfacedetection.train, which produced this model, is BSD-3-Clause. Every
link in this chain is permissive.
SFace#
What it is. “Model files encode MobileFaceNet instances trained on the
SFace loss function”, contributed by Yaoyao Zhong, ONNX conversions by
Chengrui Wang, from Zhong’s SFace paper (arXiv:2205.12010). It produces a
recognition embedding and OpenCV exposes cosine and L2 matching through
FaceRecognizerSF.match.
Accuracy, from the model card’s evaluation table:
| Model | Accuracy |
|---|---|
| SFace | 0.9940 |
| SFace block-quantized | 0.9942 |
| SFace int8 quantized | 0.9932 |
The card labels the column only “Accuracy” and does not name the dataset;
opencv_zoo/tools/eval lists LFW among its supported datasets alongside
ImageNet, WIDERFace, ICDAR, IIIT5K and Mini Supervisely, and LFW is the
only face verification set there. Read it as an LFW-family figure and
attribute it to the zoo’s own tooling, not to an independent evaluation.
Against dlib’s 99.38 and InsightFace buffalo_l’s 99.83, 99.40 puts SFace between them on the benchmark all three saturate. What SFace does not have is a published demographic breakdown; InsightFace publishes one and it is unflattering, which is a point in InsightFace’s favor on transparency even as it counts against it on the metric.
Alignment dependency, stated on the card: SFace supports “5-landmark warping for now”, and the demo uses YuNet as the detector because YuNet emits the five landmarks the warp needs. The two are designed as a pair.
License. models/face_recognition_sface/LICENSE is the Apache License
2.0.
Haar Cascades: Still There, Still 2001, and Not Under OpenCV’s License#
cv2.CascadeClassifier with haarcascade_frontalface_default.xml is the
oldest face detector in wide use and it still ships. The XML’s own header
identifies it: “Stump-based 24x24 discrete(?) adaboost frontal face
detector. Created by Rainer Lienhart.”
The licensing detail worth knowing: the cascade XML does not carry OpenCV’s Apache-2.0. Its header is an
“Intel License Agreement For Open Source Computer Vision Library Copyright (C) 2000, Intel Corporation, all rights reserved.”
which is a 3-clause-BSD-shaped grant with a no-endorsement clause naming
Intel. Permissive, redistributable, and a distinct notice from the one in
LICENSE — which matters if you are generating an attribution file
mechanically from the repository license.
The reason to still reach for Haar is that it needs no ONNX runtime, no DNN module, and about 1 MB, and it runs on anything. The reason not to is that YuNet is 227 KB, more accurate at every difficulty level, and in the same library. On any target that can run the DNN module, Haar’s remaining argument is inertia.
S1’s “70-85% accuracy” for Haar has no cited source and is not verified here; the directionally correct statement — Haar is materially worse than every DNN detector in this survey, and produces many more false positives — does not need a number.
The res10 SSD DNN Detector#
cv2.dnn with the Caffe res10_300x300_ssd_iter_140000 model is the
“OpenCV DNN” of S1’s table. It is a real option, it predates YuNet, and
it is what a large body of tutorials loads. Its practical position in 2026
is that YuNet does the same job in 227 KB instead of ~10 MB, with landmarks
the SSD does not produce, through a first-class API instead of manual blob
preprocessing. Prefer YuNet in new code.
S1’s “85-95%” for this detector is likewise uncited and not verified.
Two Release Lines, and What pip install opencv-python Gets You#
This is the packaging fact to know in 2026. PyPI holds both:
- 5.0.0.93, uploaded 2026-07-02
- 4.14.0.94, uploaded 2026-07-29 — later, on the 4.x line
opencv/opencv’s default branch is now 5.x. A plain
pip install opencv-python resolves to the highest version, which is 5.x,
and 5.x changed the DNN engine — which is what the YuNet model card’s
OPENCV_FORCE_DNN_ENGINE=4 note is about. A project that wants stability
should pin the line it tested.
Wheel coverage is broad and is the counterweight to dlib: cp37-abi3
wheels for macOS arm64 and x86_64, manylinux2014 and manylinux_2_28 on
x86_64 and aarch64, and Windows 32- and 64-bit, 34-74 MB each, plus an
82 MB sdist. One ABI3 wheel per platform covers every Python from 3.7 up.
Note the variant trap: opencv-python, opencv-contrib-python,
opencv-python-headless and opencv-contrib-python-headless all provide
the cv2 module and must not be co-installed. MediaPipe depends on
opencv-contrib-python; a project that asked for opencv-python and then
installs MediaPipe ends up with both.
Governance#
The healthiest profile in the survey:
- Open pull requests: 208
- Pull requests merged since 2026-02-25: 600
- Last commit: 2026-08-25, on the day of this pass
- 90,601 stars, 56,972 forks
A near-3:1 merge-to-open ratio over six months is the shape ADDING-RESEARCH
describes for a live project. opencv_zoo is quieter — last commit
2026-05-28, 31 open issues — which is appropriate for a model repository
that mostly needs new models rather than continuous work.
What S1 Got Wrong About OpenCV#
- “Recognition: weak” has been out of date since OpenCV 4.5.4 shipped
FaceRecognizerSF. SFace is a modern embedding model with an Apache-2.0 license, in the core library. - YuNet is treated as a passing mention. It is the best-licensed detector in the category and one of the smallest, at 227 KB.
- “OpenCV Haar: ~1 MB, Apache 2.0” — the cascade XML carries an Intel License Agreement, not Apache-2.0.
- “70-85%” and “85-95%” accuracy bands are uncited. YuNet’s WIDER FACE numbers are published and measurable; use those.
- S1’s “Very actively maintained (2024-2025)” is right and understates the change: OpenCV 5.0 shipped in 2026 and the default branch moved.
- The “Beginner Developer: start with OpenCV Haar Cascades” advice in the synthesis sends new readers to a 2001 detector when a better one with a nicer API is in the same import.
Sources#
- opencv-python on PyPI — 5.0.0.93 and 4.14.0.94, wheel matrix
- opencv/opencv — repository state, default branch
5.x,data/haarcascades/haarcascade_frontalface_default.xml - opencv_zoo YuNet model card — accuracy tables, size limits, OpenCV 5 note, MIT LICENSE
- opencv_zoo SFace model card — accuracy table, 5-landmark warping, Apache-2.0 LICENSE
- opencv_zoo eval tooling — supported datasets
- ShiqiYu/libfacedetection — 3-clause BSD upstream
- Model file sizes measured by download from
media.githubusercontent.comon 2026-08-25: YuNet 2023mar 232,589 bytes; YuNet int8 100,416 bytes; SFace 2021dec 38,696,353 bytes - pypistats: opencv-python
S2 Verdict: How These Options Differ#
Date: 2026-08-25.
The Mechanism#
Every option in this category is a chain of three artifacts:
package → weights → training dataand the license attaches at each link independently. That is the whole technical finding of this pass, and it survives contact with every project in the survey:
- MediaPipe — Apache-2.0 code, Google-published model cards, no research-only restriction found. The chain is clean end to end.
- OpenCV — Apache-2.0 code, MIT YuNet weights, Apache-2.0 SFace weights, an Intel-licensed Haar cascade. Clean, with a notice quirk.
- dlib — BSL-1.0 code, a CC0 model repository, and one model inside it whose author writes that it “can’t be used in a commercial product” because of the dataset it was trained on. The break is at the third link.
- InsightFace — MIT code, and every model research-only by the project’s own three-times-repeated statement. The break is at the second link, and the project is explicit about it.
- retina-face — MIT code over weights that entered the chain from InsightFace. The break, if there is one, is inherited and undocumented.
- DeepFace — MIT code over whichever of ten detectors and ten recognizers you name. The license of the resulting system is set by two string arguments, and the README says so.
A dependency scanner reading setup.py files sees MIT, MIT, MIT,
Apache-2.0, BSL-1.0 and reports a clean tree. It is reading the first link.
The Second Split: Three Jobs, Not One#
Detection, landmarks and recognition are three different operations with three different benchmark families, three different model families, and — as S4 develops — three different regulatory profiles.
No option in the survey leads at all three:
- Detection: RetinaFace’s lineage leads on accuracy; YuNet leads on size-per-point-of-accuracy and on license; MediaPipe leads on on-device platform reach.
- Landmarks: MediaPipe leads outright at 478 3D points with blendshapes, and is permissive.
- Recognition: InsightFace leads on the hard benchmarks and is research-only; SFace is the best-licensed real option; dlib’s ResNet is the mature one.
The practical consequence is that most production face pipelines are two projects, not one, and the pairing question is whether the detector emits the landmarks the encoder’s alignment step wants. YuNet + SFace are designed as a pair. RetinaFace + ArcFace are designed as a pair. Mixing across pairs means writing the warp yourself, and dlib’s own model README warns about exactly this failure mode for its shape predictors.
What S1 Got Wrong#
S1’s individual library files are competent. Its comparison table and several of its verdicts have decayed or were wrong on arrival. Collected:
Factual errors#
- “buffalo_sc: High accuracy, larger model” — backwards. It is the
16 MB pack, the smallest in the family, with
buffalo_saccuracy. - “OpenCV recognition: weak” — out of date since OpenCV 4.5.4.
cv2.FaceRecognizerSFruns SFace at 0.9940, Apache-2.0. - “OpenCV Haar: Apache 2.0” — the cascade XML carries an Intel License Agreement, not OpenCV’s Apache-2.0.
- “Dlib: Boost, ✓ Free commercial use” as a single row — the 68-point landmark model that S1’s own code samples load carries an author’s note saying it cannot be used in a commercial product.
- “RetinaFace: MIT (implementations), ✓ Free” — the MIT covers the re-implementation code. The weights came from InsightFace.
- “InsightFace: Free (non-commercial)” — right about the models, wrong about the code, which is MIT with no commercial limit. The distinction is the decision.
- “MediaPipe: 468 points” — the maintained Tasks API and Google’s current documentation say 478.
- "~150 MB package" for dlib — the distribution is a 3.3 MB source tarball with no wheels at all.
Claims with no source, recorded here as not verified#
- MediaPipe “99.3% (comparative study)” — study unnamed, not a WIDER FACE figure, and tabulated against WIDER FACE numbers.
- MediaPipe “60-100 FPS” CPU.
- OpenCV Haar “70-85%” and OpenCV DNN “85-95%”.
- MTCNN “97.56% AUC (2016)” and “5-15 FPS”.
- Dlib CNN “1-3 FPS”, Dlib HOG “30+ FPS”.
- RetinaFace “GPU: 30-50 FPS”.
- InsightFace “buffalo_l: 99.88% detection success on LFW” — no corresponding figure in the model zoo.
- Face++ “$100+/day” and “99%+” accuracy.
- “Self-hosted breaks even at ~100K faces/month.”
None of these are necessarily false. None could be traced to a source, and every one of them appears in a table where the reader is invited to subtract it from a neighbor.
Structural problems#
- One table for three jobs. WIDER FACE AP and LFW accuracy in adjacent columns.
- RetinaFace and InsightFace presented as competitors. They are one lineage from one lab; the RetinaFace paper’s code link points into the InsightFace repository.
- The wrapper tier is absent.
face_recognitionhas 56,682 stars — more than any project S1 covers — and no release since 2020. DeepFace has 23,337 stars, 487,770 monthly downloads, and is actively maintained. Neither appears. - YuNet and SFace are absent, and they are the only detection + recognition pair in the category with unambiguous permissive licenses on the weights.
- “GDPR-compliant” applied to on-device processing. Local processing changes the transfer and processor questions; it does not change the Article 9 lawful-basis question.
- The demographic performance data InsightFace publishes about its own models is not mentioned, though it sits on the same table as the 99.83 LFW figure S1 does quote.
Verdicts that have expired since January 2025#
- InsightFace shipped 1.0 and 1.0.1 in May 2026 after three years of silence, gaining wheels, losing its PyPI license classifier, and adding an enterprise evaluation GUI whose documentation names “a commercial model license” as the destination.
- InsightFace added a licensing contact for
buffalo_lon 2025-11-24 and announced a Rekognition-alternative server on 2026-07-27. - MediaPipe shipped 1.0.0 on 2026-07-27.
- OpenCV shipped 5.0 and moved its default branch to
5.x, changing the DNN engine in a way that affects which YuNet ONNX file to load. facenet-pytorchwent dormant: last commit 2024-08-02, 7 patches waiting, 0 merged.ipazc/mtcnnhas been untouched since 2024-10-08.
What Merits an S3 Persona#
The technical differences above cluster into a small number of situations where a different option wins. Those become the S3 personas:
- The on-device geometry case — AR, avatars, expression, gaze. One answer, and it is not close.
- The permissive-stack product case — a commercial product that needs detection and recognition and cannot negotiate model licenses.
- The research and benchmarking case — accuracy is the objective and the non-commercial restriction is satisfied by the use.
- The constrained-hardware presence case — detection only, kilobytes, no identity.
- The regulated-deployment case — the compliance surface drives the architecture before any model is chosen.
- The inheritance case — code exists, it imports something dormant.
S3 develops these as people with problems rather than as configurations.
What This Pass Could Not Establish#
Recorded so the next refresh knows where to start:
- Whether the RetinaFace weights redistributed through
retina-facecarry the InsightFace non-commercial restriction. The code chain is MIT; no relicensing statement for the weights was found anywhere in it. - Face++ per-call pricing. The table renders client-side.
- Independent accuracy numbers for any open-source model here. None of them participate in NIST FRTE, so every figure in this survey is self-reported by the project that published the model.
- Latency on comparable hardware. No cross-project benchmark exists that this pass could verify, and running one is a build, not a survey.
RetinaFace and MTCNN: The Re-implementation Layer#
These two are grouped because they share a shape that nothing else in this survey has. Neither is a maintained product. Each is a paper whose weights reached practitioners through a chain of community re-implementations, and the thing a reader installs is several hops downstream from the thing the accuracy number describes.
Everything below was checked on 2026-08-25.
RetinaFace#
The Paper, and Where the 91.4% Comes From#
RetinaFace is Deng, Guo, Ververas, Kotsia and Zafeiriou, “RetinaFace: Single-stage Dense Face Localisation in the Wild”, arXiv:1905.00641, submitted 2019-05-02, revised 2019-05-04. The abstract states the claim S1 carries:
“(3) On the WIDER FACE hard test set, RetinaFace outperforms the state of the art average precision (AP) by 1.1% (achieving AP equal to 91.4%).”
Three qualifications belong with that number every time it is quoted.
It is the paper authors’ own measurement, and those authors are the
InsightFace group — the abstract’s own code link is
github.com/deepinsight/insightface/tree/master/RetinaFace. RetinaFace is
not an independent competitor to InsightFace; it is InsightFace’s detector,
which is why the buffalo packs list “RetinaFace-10GF” in their detection
column.
It is the hard test set, not the validation set. YuNet’s 0.7503 hard AP is a validation-set figure. The two sets are different and the numbers are not directly comparable, which is the sort of thing a survey should say out loud rather than tabulate side by side.
It is the paper’s best configuration, a heavy backbone with the
self-supervised mesh decoder branch — not the MobileNet-0.25 variant people
actually deploy. The most-forked PyTorch re-implementation,
biubug6/Pytorch_Retinaface, states its own result in its repository
description: “Retinaface get 80.99% in widerface hard val using
mobilenet0.25.” That is an eleven-point gap between the paper’s headline and
the small model most projects run, and it is published by the
re-implementation itself.
The abstract’s other verified figure is worth carrying because it is the one about identity: “On the IJB-C test set, RetinaFace enables state of the art methods (ArcFace) to improve their results in face verification (TAR=89.59% for FAR=1e-6).”
What You Get If You pip install retina-face#
retina-face 0.0.18, uploaded 2026-06-01. Repository
serengil/retinaface — MIT, 2,027 stars, 1 open issue, 0 open pull
requests, 1 merged since 2026-02-25, last commit 2026-06-01.
The README documents the provenance chain without being asked:
“RetinaFace is the face detection module of insightface project. The original implementation is mainly based on mxnet. Then, its tensorflow based re-implementation is published by Stanislas Bertrand. So, this repo is heavily inspired from the study of Stanislas Bertrand. Its source code is simplified and it is transformed to pip compatible but the main structure of the reference model and its pre-trained weights are same.”
Emphasis added. The weights are the same weights. The code path is:
InsightFace RetinaFace (mxnet)
→ StanislasBertrand/RetinaFace-tf2 (MIT code, last push 2024-05-11)
→ serengil/retinaface (MIT code)
→ retinaface.h5, re-hosted at
github.com/serengil/deepface_models/releases/download/v1.0/confirmed in retinaface/model/retinaface_model.py, which downloads that
file to ~/.deepface/weights/retinaface.h5.
Every code repository in that chain is MIT. The weights entered the chain from InsightFace, whose model zoo states that all its models “are available for non-commercial research purposes only”. Three MIT re-hostings do not relicense an upstream artifact.
This survey does not resolve that question — it is a legal question about a specific file’s provenance, and the answer is not published anywhere this pass could read. What it records is that the chain does not terminate in a permissive grant on the weights, and that S1’s licensing table entry — “RetinaFace: MIT (implementations), ✓ Free commercial use” — describes only the code repositories and not the file that does the detecting.
Maintenance#
serengil/retinaface is small, tidy and alive: 0 open PRs, 1 open issue,
a release in June 2026. Its author, Sefik Ilkin Serengil, also maintains
DeepFace (see wrappers.md), and the two projects share the
~/.deepface/weights/ cache and the serengil/deepface_models release
host.
biubug6/Pytorch_Retinaface, the 2,976-star PyTorch implementation that
much research code imports, has a last commit of 2020-04-20. It is
dormant, not archived, and 148 issues are open against it. Anyone
inheriting a codebase that vendored it is maintaining that code themselves.
MTCNN#
The Method#
MTCNN is Zhang, Zhang, Li and Qiao, “Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks” (2016). Three small networks — P-Net, R-Net, O-Net — run over an image pyramid, each stage rejecting candidates and refining boxes, with the final stage also producing five landmarks. It was a real advance in 2016 and it is comprehensively superseded.
The Package#
mtcnn 1.0.0, uploaded 2024-10-08. Repository ipazc/mtcnn — MIT,
2,484 stars, 527 forks, 28 open issues, 0 open pull requests, 0 merged
since 2026-02-25, last commit 2024-10-08 (the 1.0.0 merge itself).
Downloads: 256,107 in the 30 days to 2026-08-25.
That profile needs care. Zero merges over six months is one half of the stall signal; zero open pull requests is the other half missing. ADDING-RESEARCH’s test fires on the ratio — patches arriving and nobody merging them. Here nothing is arriving either. This is not a stalled project; it is a finished one. A 2016 cascade detector with a clean 1.0.0 release and an empty queue is a reasonable thing for a maintainer to stop touching.
The operational risk is not abandonment-by-neglect but the fact that nothing will be fixed. The package’s history — 0.1.1 in 2021, then nothing until 1.0.0 in 2024 — shows a maintainer who returns for major work occasionally, which is not the same as a maintenance commitment.
S1’s “2 MB” model size is consistent with the three cascade networks and is the one thing MTCNN is still best at in this survey. Its “5-15 FPS” CPU figure has no cited source and is not verified; the cascade design makes MTCNN’s speed strongly dependent on image size, pyramid scale factor and how many candidates survive each stage, so a single FPS number for it is less meaningful than for a single-shot detector.
S1’s “97.56% AUC (2016)” is likewise uncited and is an FDDB-style metric rather than a WIDER FACE AP; it cannot be compared with any other row in S1’s table.
Where MTCNN Still Turns Up#
Mostly inside other things. facenet-pytorch bundles an MTCNN
implementation as its detector; DeepFace offers both mtcnn and
fastmtcnn (the facenet-pytorch one) as detector backends. A reader who
believes they are not using MTCNN may be.
Reading These Two Together#
The lesson these two share is about how accuracy claims travel. S1’s master table puts “RetinaFace 91.4%” and “MTCNN 97.56% AUC” in the same column, which invites the conclusion that MTCNN detects better. It does not. The numbers are from different benchmarks, different years, different metrics, and — in RetinaFace’s case — from a model configuration that almost nobody runs.
The practical position in 2026:
- If you want the RetinaFace lineage’s accuracy, use InsightFace directly and resolve its license. Going through a re-implementation gets you the same weights with a friendlier README and no better license answer.
- If you want a small permissive detector, YuNet is 227 KB, MIT, and scores better on WIDER FACE hard than the MobileNet RetinaFace variant.
- If you are on MTCNN, nothing is going to break, and nothing is going to improve. Treat it as a frozen dependency and check whether YuNet replaces it in a morning.
Sources#
- RetinaFace, arXiv:1905.00641 — 91.4% WIDER FACE hard test set; IJB-C TAR=89.59% at FAR=1e-6; code link into insightface
- retina-face on PyPI — 0.0.18, MIT classifier
- serengil/retinaface README — provenance chain, “pre-trained weights are same”
retinaface/model/retinaface_model.py— weight download URL- StanislasBertrand/RetinaFace-tf2 — MIT, last push 2024-05-11
- biubug6/Pytorch_Retinaface — “80.99% in widerface hard val using mobilenet0.25”, last commit 2020-04-20
- mtcnn on PyPI — 1.0.0, MIT
- ipazc/mtcnn — repository state
- InsightFace Model Zoo — “ALL models are available for non-commercial research purposes only”
The Wrapper Tier: face_recognition, DeepFace, facenet-pytorch#
S1 compares six models and two hosted services. It does not cover the layer most people actually install, which is a wrapper: a package that hides the detector, the aligner and the encoder behind one function call and downloads whatever weights it needs on first use.
The wrapper tier deserves its own file because it is where the license chain gets longest and where the maintenance picture is worst. Between them these three packages account for 810,311 PyPI downloads in the 30 days to 2026-08-25 — more than dlib, MTCNN and retina-face combined.
Everything below was checked on 2026-08-25.
face_recognition (ageitgey): The Most-Starred Dead Project in the Category#
Package: face-recognition 1.3.0, uploaded 2020-02-20.
Repository: ageitgey/face_recognition — MIT, 56,682 stars,
13,699 forks, 832 open issues, not archived.
Downloads: 186,937 in the 30 days to 2026-08-25.
This is the most-starred repository in this entire survey — more stars than InsightFace, more than dlib, more than MediaPipe — and its last release was six and a half years ago.
The stall test, applied:
- Open pull requests: 57
- Pull requests merged since 2026-02-25: 1
- Last commit on
master: 2026-06-25, and that commit isMerge pull request #1672 from 17X61/auto-contrib/1670-lfw-link— a link fix.
pushed_at is recent, which is how a project like this passes a shallow
freshness check. The substance is a single merged link repair against 57
waiting patches and 832 open issues. S1’s own “Updates & Maintenance”
section does not list face_recognition at all, so S1 does not get this
wrong so much as leave a 56,000-star project out of a survey of the
category.
What it actually is. A thin, well-designed Python API over dlib. It
uses dlib’s HOG or CNN detector, dlib’s shape predictor, and dlib’s
dlib_face_recognition_resnet_model_v1 encoder. Its 99.38% LFW claim is
dlib’s number, measured by Davis King on Davis King’s model.
Consequences for a 2026 reader:
- It inherits dlib’s build problem.
face_recognitiondepends ondlib, andpip install dlibcompiles from source (seedlib.md). Every “face_recognition won’t install” thread is a CMake thread. - It inherits dlib’s license chain. If the model it loads is the 68-point shape predictor, the note in dlib’s own model README applies.
- A 2020 dependency pin against a 2026 NumPy and Python is where the breakage lives, and nobody is merging the fixes: 57 of them are open.
- It cannot be a compliance answer. Six years without a release means six years without a security fix in the packaging.
The forks are the ecosystem’s response: 13,699 of them, and the distributions that ship “face_recognition” as a system package are generally carrying patches.
Verdict: an excellent API on a stopped project. If you are starting
today, the code you would write against face_recognition is nearly the
same code you would write against FaceDetectorYN + FaceRecognizerSF,
with a live library and a clean license underneath.
DeepFace (serengil): Alive, Broad, and Explicit About the Trap#
Package: deepface 0.0.100, uploaded 2026-05-09.
Repository: serengil/deepface — MIT, 23,337 stars, 3,168 forks,
9 open issues, last commit 2026-08-24.
Downloads: 487,770 in the 30 days to 2026-08-25.
Stall test:
- Open pull requests: 2
- Merged since 2026-02-25: 11
- Last commit 2026-08-24 (
discard mypy attribute defined errors (#1622))
Nine open issues against 23,337 stars is the tidiest tracker in this survey. DeepFace is maintained.
What it is. A framework that presents one API — DeepFace.verify,
DeepFace.find, DeepFace.analyze, DeepFace.represent — over a
swappable matrix of detector backends and recognition models. You pick a
detector_backend and a model_name; DeepFace fetches the weights and
runs the pipeline.
Why it belongs in this survey’s license argument. DeepFace’s own README states the situation more clearly than any other document this pass read:
“DeepFace wraps some external face recognition models: VGG-Face, Facenet (both 128d and 512d), OpenFace, DeepFace, DeepID, ArcFace, Dlib, SFace, GhostFaceNet and Buffalo_L. … Similarly, DeepFace wraps many face detectors: OpenCv, Ssd, Dlib, MtCnn, Fast MtCnn, RetinaFace, MediaPipe, YuNet, Yolo and CenterFace. Finally, DeepFace is optionally using face anti spoofing to determine the given images are real or fake. License types will be inherited when you intend to utilize those models. Please check the license types of those models for production purposes.”
Emphasis in the original. That is the chain rule of this whole category, written by a maintainer in his own README.
Working through what it means for two backend choices:
model_name="Buffalo_L"loads InsightFace’sbuffalo_l, which InsightFace states is for non-commercial research only. The DeepFace package around it is MIT; the model is not.detector_backend="yolov8"loads a model fromultralytics/ultralytics, which is AGPL-3.0. An AGPL detector inside a network service is a materially different obligation from an MIT one, and DeepFace’s MIT license does not change it.
Meanwhile detector_backend="yunet" and model_name="SFace" are the MIT
and Apache-2.0 pair from opencv_zoo, and detector_backend="opencv" is
the Intel-licensed Haar cascade. The same package, the same API, the same
two lines of code, produce a permissive stack or an AGPL one or a
research-only one depending on two string arguments. That is the sharpest
illustration of this survey’s organizing point that the category offers.
What DeepFace is good at, beyond the pipeline: DeepFace.analyze
provides age, gender, emotion and race attribute models that the README
says were “trained on the backbone of VGG-Face with transfer learning” —
the self-hosted answer to the attribute analysis that S1 sends readers to
the cloud APIs for. Whether you should be inferring those attributes at all
is a question S4 takes up, and the EU AI Act has a view on two of them.
facenet-pytorch (timesler): Dormant#
Package: facenet-pytorch 2.6.0, uploaded 2024-04-29.
Repository: timesler/facenet-pytorch — MIT, 5,162 stars, 1,000 forks,
85 open issues, last commit 2024-08-02.
Downloads: 135,604 in the 30 days to 2026-08-25.
Stall test:
- Open pull requests: 7
- Merged since 2026-02-25: 0
Two years without a commit, seven patches waiting, no release since April 2024. This is a dormant project, and it is where a large amount of research code gets both its MTCNN detector and its InceptionResnetV1 embeddings (pretrained on VGGFace2 and CASIA-WebFace, per the repository description).
The PyTorch coupling is the specific risk. A pinned-torch package that
nobody is updating becomes uninstallable against a current torch on a
schedule set by someone else. mtcnn at least has no framework
underneath it worth breaking; facenet-pytorch does.
DeepFace exposes this project’s detector as fastmtcnn, so the dormancy
propagates into a maintained package as one backend option.
The Pattern#
| face_recognition | DeepFace | facenet-pytorch | |
|---|---|---|---|
| Stars | 56,682 | 23,337 | 5,162 |
| Last release | 2020-02-20 | 2026-05-09 | 2024-04-29 |
| Last commit | 2026-06-25 (link fix) | 2026-08-24 | 2024-08-02 |
| Open PRs / merged since 2026-02-25 | 57 / 1 | 2 / 11 | 7 / 0 |
| Open issues | 832 | 9 | 85 |
| Package license | MIT | MIT | MIT |
| Model licenses | dlib’s chain | whatever backend you name | VGGFace2 / CASIA-WebFace terms |
| 30-day downloads | 186,937 | 487,770 | 135,604 |
Three MIT wrappers. One alive. All three download weights whose terms are set by someone else, and only one of them tells you so.
Star count is the least informative column in that table, and it is the one a reader picking a library sees first. The two dormant projects hold 61,844 stars between them against DeepFace’s 23,337.
Sources#
- face-recognition on PyPI — 1.3.0, 2020-02-20
- ageitgey/face_recognition — repository state, PR counts, last commit
- deepface on PyPI — 0.0.100, 2026-05-09
- serengil/deepface README — wrapped model list and the license-inheritance paragraph
- ultralytics/ultralytics — AGPL-3.0
- facenet-pytorch on PyPI — 2.6.0, 2024-04-29
- timesler/facenet-pytorch — repository state
- pypistats — 30-day download counts to 2026-08-25
S3: Need-Driven
S3: Need-Driven Discovery — Approach#
Date: 2026-08-25.
Why the Personas Split This Way#
S1 organized its use cases by application: security systems, photo organization, AR filters, attendance tracking, age verification, video conferencing, retail analytics. That is a list of products, and it produces the same recommendation for several of them because the products are not what differs.
What differs is a small set of constraints that are settled before anyone looks at an accuracy number:
- Which of the three jobs do you need? Detection, landmarks, or identity. Most readers need one. A reader who needs identity has a different problem from a reader who needs a bounding box, in every dimension including the legal one.
- Can you accept a research-only model? This has a yes/no answer and it removes the highest-scoring options from the table when the answer is no.
- Where does the code run? A browser, a phone, a Raspberry Pi, a GPU server, someone else’s cloud.
- Are you enrolling people? Storing a template that identifies a named individual is a different undertaking from computing one and throwing it away.
- Are you starting, or inheriting? A reader with a working
face_recognitioncodebase has a different question from a reader with an empty repository.
The personas below are the distinct combinations. Each one is a person with a problem, not a configuration.
The Personas#
| File | Who | The constraint that decides it |
|---|---|---|
use-case-ar-developer.md | Ships face effects in an app | Needs dense 3D geometry on-device, three platforms |
use-case-product-engineer.md | Ships a commercial product with face matching | Cannot negotiate a model license |
use-case-research-lab.md | Publishes on face recognition | Accuracy is the objective; non-commercial is satisfied |
use-case-embedded-integrator.md | Detection on constrained hardware | Kilobytes, no identity, no network |
use-case-compliance-owner.md | Answers for a biometric deployment | The regulatory surface comes before the model |
use-case-inheritor.md | Maintains code someone else wrote | The dependency is dormant and the build broke |
Six personas, one of which — the compliance owner — is not a developer at all, and is in the survey because in this category that person often holds the veto.
What Is Not Here#
“Beginner learning computer vision” is not a persona in this survey. S1 had one and recommended Haar cascades, which sends a newcomer to a 2001 detector so they can be disappointed by it. Anyone learning should use the same tools as anyone shipping; the tools are not harder.
“Startup MVP” is not a persona either. The MVP-versus-scale framing in S1 assumes the only axis is cost, and in this category it is not — the question of whether biometric data leaves your infrastructure does not get easier to answer later, and a hosted API that indexes a gallery on your behalf is a migration when you change your mind.
Surveillance and law-enforcement identification are not developed as personas. They exist, they are a large part of this market, and the regulatory material in S4 is where a reader in that position should start rather than in a library comparison.
How to Read the Personas#
Each file states who the person is, what they are stuck on, which options survive their constraints and which are eliminated and why, what they give up, and the condition under which the recommendation flips. The recommendation is conditional in every case, and where a persona’s answer is the same as another’s, that is said rather than hidden behind different wording.
S3 Verdict: Who Should Use What#
Date: 2026-08-25.
The Map#
| Persona | Recommendation | The constraint that decided it |
|---|---|---|
| AR developer | MediaPipe Face Landmarker | 478 3D landmarks + blendshapes + pose matrix, first-party on three platforms, permissive. Nothing else does this job. |
| Product engineer | cv2.FaceDetectorYN + cv2.FaceRecognizerSF | The only detection + recognition pair whose weights carry a license file permitting commercial use |
| Research lab | InsightFace | The only project publishing results on the benchmarks that separate models; the non-commercial restriction is satisfied by the use |
| Embedded integrator | YuNet int8 (98 KB), or dlib HOG for a zero-artifact build | Flash budget, no network, model redistributed inside a device |
| Compliance owner | Not a library — three questions before the model | Storage, transfer, and verification-versus-identification decide more than any benchmark |
| Inheritor | Swap the detector to YuNet; leave the encoder alone unless forced | Stored templates are in the old encoder’s embedding space |
What Repeats Across Personas#
YuNet appears in three of the six answers. For a 227 KB detector that S1 mentions twice in passing and never analyzes, that is the finding of this survey compressed into one observation. It is MIT at the weights, it is inside a library nearly everyone already has, and it is competitive on the benchmark it is measured against.
The encoder is the irreversible choice. Two separate personas — the research lab and the inheritor — arrive independently at the same discipline: keep the encoder behind an interface. Detectors are interchangeable in an afternoon. Encoders are not interchangeable at all once templates are stored, because a template is meaningless outside the model that produced it. This is the one architectural rule this survey would put in front of every reader.
Accuracy decides almost nothing. In five of the six personas the recommendation is settled by license, platform reach, model size or maintenance before any benchmark is consulted. The only persona where an accuracy number is decisive is the research lab, and there the decisive numbers are IJB-C and MR-ALL rather than the LFW figure S1 leads with.
Detection-only is an underused answer. Two personas need it, and a third — the compliance owner — spends most of their analysis trying to establish whether a proposed feature needs it. Delivering a feature with detection alone avoids the entire biometric-template apparatus. Engineers reach for recognition by default because it is the impressive part.
Where the Personas Disagree#
On InsightFace. The research lab’s first choice is the product engineer’s elimination, on identical facts. Both are right. The license is not a defect; it is a term, and it maps onto one use and not the other. A survey that recommended InsightFace or refused to would be wrong for half its readers.
On MediaPipe. The AR developer’s only viable option is invisible to the product engineer, because MediaPipe has no recognition model at all. Two readers, one project, and it is either the whole answer or not an answer.
On “just use the cloud”. The product engineer treats Amazon Rekognition as a reasonable fallback when there is no ML capacity and volume is low. The compliance owner treats sending face images to a third party as the decision that needs the most scrutiny. Both positions are sound and the disagreement is the point: this trade cannot be resolved inside engineering.
The Persona S1 Had That This Survey Dropped#
S1 recommended Haar cascades to the “Beginner Developer” and told them to avoid “RetinaFace training, InsightFace setup complexity”. The intent was kind and the result is that a newcomer meets a 2001 detector, gets poor results, and concludes face detection is hard.
cv2.FaceDetectorYN is not harder to call than cv2.CascadeClassifier. It
is better, smaller, better licensed, and in the same import. There is no
beginner tier in this category — the good tools are the easy ones.
What S3 Assumes, and Where It Could Be Wrong#
These personas are constructed from the technical differences S2 established, not from interviews. Two of them make claims that would be worth testing against people who actually hold the job:
- The compliance owner is assumed to hold a veto. In organizations where they do not, the ordering in that persona is wrong and the analysis arrives after the architecture.
- The product engineer is assumed to be unable to negotiate a model license. For a company where face matching is the product rather than a feature, that assumption inverts, and the persona’s elimination of InsightFace goes with it.
Both are flagged rather than hidden, because a persona is a hypothesis about who is reading.
Use Case: The AR Developer#
Who Needs This#
An engineer at a small studio building face effects into a consumer app — virtual try-on for eyewear, an avatar that mirrors the user’s expression, a filter that tracks a face through motion. The app ships on iOS and Android, and marketing wants a web demo that works without an install.
The team is three people. Nobody has trained a model. The face work is one feature among many.
Why They Need It#
The effect has to sit on the face and stay there. A bounding box is useless — you cannot anchor a pair of glasses to a rectangle. What the renderer needs is a dense mesh with depth, updated every frame, plus enough information about expression to drive an avatar’s face and enough about head pose to place a 3D object in the scene.
The frame budget is the hard constraint. At 30 fps there are 33 milliseconds for everything: camera, tracking, the app’s own rendering, and the UI. Face tracking gets a fraction of that, on a mid-range phone, without draining the battery.
And it has to be the same feature on three platforms. A studio of three cannot maintain a Core ML pipeline, a TFLite pipeline and a WebAssembly pipeline that produce subtly different landmarks.
Requirements#
- Dense 3D landmarks, not five points and not a box
- Expression coefficients that can drive a rig directly
- A head-pose transform, so 3D objects can be placed without solving for it
- Sub-10 ms per frame on a mid-range phone
- First-party iOS, Android and web bindings from one project
- A license that survives a commercial app store release
- No identity. The app never needs to know who the user is, and the studio does not want to acquire that obligation
Which Options Survive#
MediaPipe Face Landmarker, and nothing else is close.
It is the only option in the survey that produces 478 3D landmarks, 52 blendshape coefficients, and a facial transformation matrix from one model bundle, with first-party Android, iOS and Web APIs maintained by the same team. The blendshapes are the piece that has no substitute: they map onto the same coefficient space that character rigs are authored against, so an avatar is a wiring exercise rather than a research project. The transformation matrix removes the pose-solving step entirely.
The published latency supports the budget. Google’s own figure for the
BlazeFace short-range detector is 2.94 ms CPU on a Pixel 6, and the
detector is the front of the pipeline; the mesh and blendshape models run
per detected face with num_faces defaulting to 1. Google publishes no
full-bundle latency, so the studio should measure — but the shape of the
budget is favorable and the platform bindings make measuring cheap.
The license is the quiet reason it wins. MediaPipe is the option in this survey where the chain from package to weights to documentation turned up no research-only restriction anywhere. An app store release is not a license question.
What Is Eliminated, and Why#
- dlib’s 68-point predictor. Fewer points, 2D only, no blendshapes, no mobile bindings — and the model’s own author writes that it “can’t be used in a commercial product”. Three disqualifications, any one of which would be enough.
- InsightFace’s 2d106/3d68. More points than dlib, still fewer than MediaPipe, no expression coefficients, and non-commercial by the project’s own statement.
- YuNet, MTCNN, RetinaFace. Five points. They are detectors. A five-point landmark set aligns a face for recognition; it does not anchor geometry.
- Face++ dense landmarks. Technically capable and 3D reconstruction is a listed product, but it is a network round trip per frame. At 30 fps that is 30 API calls per second per user, which is neither affordable nor fast enough.
- Anything hosted. Same reason. Real-time face tracking is an on-device problem.
What They Give Up#
Recognition, entirely. MediaPipe has no embedding model. If the product later wants “remember this user’s face to load their settings”, that is a second project with a second model and — as the compliance persona develops — a different legal posture. The studio should treat that as a feature they do not have rather than a feature they will add cheaply.
Detection accuracy in hard conditions. BlazeFace short-range is optimized for a face filling a selfie-distance frame. Small faces, crowds, extreme angles and heavy occlusion are not what it is for. For an AR app where the user is holding the phone, that is the right trade.
Outside patches. MediaPipe’s development happens inside Google and syncs out; 199 pull requests are open against zero merged in six months. A bug the studio finds is a bug they wait for, or work around.
Intel Macs. mediapipe 1.0.1 publishes a macOS arm64 wheel and no
x86_64 wheel. If anyone on the team is on an older Mac, the Python
prototyping path is closed to them even though the shipping app is
unaffected.
When This Recommendation Flips#
- If the effect needs identity — a filter that recognizes a returning user — MediaPipe stops being sufficient and the product engineer’s persona applies instead, for that half.
- If the target is a desktop application with a GPU and the studio wants research-grade 3D face reconstruction rather than a mesh, InsightFace’s face3d work and the wider 3D morphable model literature are the place to look, with the non-commercial question attached.
- If the deployment is a browser only and 30 MB of WASM is too much first-load weight, a five-point detector plus hand-rolled geometry is the fallback — a real downgrade, taken for a real reason.
Decision Criteria, Compressed#
Dense 3D face geometry, on-device, cross-platform, commercially shippable: MediaPipe Face Landmarker. The question is not which is best; it is that nothing else in the category does this job.
Use Case: The Compliance Owner#
Who Needs This#
The person who signs off. A privacy counsel, a data protection officer, a security architect, or — in a small company — the founder who will be answering the questions. They are not choosing a library. They are deciding whether a proposed feature ships at all, and if so under what constraints.
Increasingly this person is also an engineering lead who has worked out that in this category the compliance surface arrives before the architecture, not after it.
Why They Need It#
Face software is the one category in this library where the choice of library is downstream of a legal analysis, and where the analysis turns on a distinction engineers routinely collapse.
Detecting that a face is present in a frame is one operation. Computing a template that identifies a specific person is a different one. The two use different models, produce different artifacts, and sit differently under every regime this survey examined. The vendors themselves draw the line: Microsoft sells face detection to anyone and gates face identification behind a vetting process. Texas’s 2026 amendment exempts AI training on biometric identifiers “unless a system is used or deployed for the purpose of uniquely identifying a specific individual”. The EU AI Act’s Annex III carves verification out of remote identification.
A compliance owner who understands that line can approve half the proposals that cross their desk quickly and route the other half properly. One who does not will either block everything or approve something they should not have.
This section reports what the rules say and where each was read. It is not legal advice, it does not predict how any rule will be applied, and a deployment decision needs counsel who has seen the specifics.
Requirements#
- Know which of the three jobs the feature actually needs
- Know whether a biometric template will be stored, and for how long
- Know where the data goes: on device, on your servers, or to a vendor
- Know which jurisdictions the users are in
- Have written answers to the notice, consent, retention and destruction questions before the model is picked
- Be able to show the provenance and license of every model in the build
The Questions That Decide It#
1. Are you storing a template?
The gap between “detect a face and draw a box” and “compute an embedding and keep it” is the gap between an image-processing feature and a biometric database. Illinois’s BIPA definitions turn on this: a “biometric identifier” is “a retina or iris scan, fingerprint, voiceprint, or scan of hand or face geometry”, and the Act’s Section 15 obligations — a published written retention policy, written notice, a written release, destruction “when the initial purpose for collecting … has been satisfied or within 3 years of the individual’s last interaction … whichever occurs first” — attach to possessing them.
A pipeline that detects, crops, blurs and discards is a different object from one that enrolls.
2. Does anything leave your infrastructure?
Sending images to a hosted API is a processor relationship and, for EU data, a transfer question. Running the model locally removes both. It does not remove the lawful-basis question: GDPR Article 9(1) makes “biometric data for the purpose of uniquely identifying a natural person” a special category whose processing “shall be prohibited” absent an Article 9(2) condition, and that applies to a model running on your own hardware.
S1’s framing of on-device processing as “GDPR-compliant” is the single most consequential error in it, because it is the sentence an engineer quotes to a compliance owner.
3. Which line of the EU AI Act does this touch?
Regulation (EU) 2024/1689 treats three biometric uses as prohibited under Article 5(1), and the two that catch ordinary product ideas are not the ones people expect:
- (f) prohibits “the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons”. Emotion inference is a checkbox in several libraries in this survey.
- (g) prohibits biometric categorisation systems “that categorise individually natural persons based on their biometric data to deduce or infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation”. The “race/ethnicity” attribute head that ships in at least one wrapper here is squarely in that description.
- (h) prohibits real-time remote biometric identification in publicly accessible spaces for law enforcement, subject to narrow exceptions for victim search, imminent threats, and suspect localisation.
Annex III then classifies as high-risk, “in so far as their use is permitted under relevant Union or national law”: remote biometric identification systems, excluding those used solely to verify that a person is who they claim to be; biometric categorisation by sensitive or protected attribute; and emotion recognition.
The verification carve-out is the practical one. Unlocking a device by matching a face against the one template on that device is verification. Scanning a crowd against a gallery is remote identification. Same encoder, different regime.
4. Who can be sued, and by whom?
The two US statutes this survey read differ on this and the difference shapes risk more than the substantive rules do. BIPA has a private right of action: Section 20 gives “any person aggrieved by a violation” a claim with liquidated damages of $1,000 for negligent violations or $5,000 for intentional or reckless ones, “or actual damages, whichever is greater”, plus fees. Texas’s CUBI has no private right of action: Section 503.001(d) provides “a civil penalty of not more than $25,000 for each violation” and “the attorney general may bring an action to recover the civil penalty.”
An amendment effective 2024-08-02 (P.A. 103-769) narrowed BIPA’s per-scan exposure: repeated collection of “the same biometric identifier or biometric information from the same person using the same method of collection” is now “a single violation … for which the aggrieved person is entitled to, at most, one recovery”, with a parallel rule for repeated disclosure. The same act added “electronic signature” to the definition of “written release”.
5. Are you training on scraped images?
Texas answered this in 2026. CUBI as amended by H.B. 149, effective 2026-01-01, adds subsection (b-1): an individual “has not been informed of and has not provided consent for the capture or storage of a biometric identifier … based solely on the existence of an image or other media containing one or more biometric identifiers of the individual on the Internet or other publicly available source unless the image or other media was made publicly available by the individual to whom the biometric identifiers relate.”
The same amendment adds an exemption at (e)(2) for “the training, processing, or storage of biometric identifiers involved in developing, training, evaluating, disseminating, or otherwise offering artificial intelligence models or systems, unless a system is used or deployed for the purpose of uniquely identifying a specific individual”, with (f) pulling a model back under the section if identifiers captured for training are later used commercially outside the exemption.
Note the relevance to this survey’s own material: dlib’s recognition model was trained in part on “a large number of images I scraped from the internet”, by the author’s own README.
Which Options Survive#
The compliance owner is not choosing a library, but the analysis eliminates some and elevates others:
Elevated by a “detection only” scope: YuNet, dlib HOG, MediaPipe’s Face Detector. If the feature can be delivered without an embedding, the whole identification analysis falls away and what remains is ordinary image processing.
Elevated by “verification, not identification”: any encoder, used against a single enrolled template held on the user’s own device. That architecture sits inside the Annex III verification carve-out and is the posture Microsoft’s own gating treats as least sensitive.
Elevated by “nothing leaves the building”: the self-hosted libraries, for the transfer and processor questions — and only for those. The lawful-basis question is unchanged.
Requires an explicit decision: any attribute inference. Age, gender, emotion and race heads ship in DeepFace and in the hosted APIs, and the EU AI Act names emotion inference and sensitive-attribute categorization directly. A team that turned these on because they were available should be asked why.
Requires a licensing paper trail regardless: every model in the build, with its license file. A compliance owner who asks “what is the license of the weights, not the package” will find that the answer differs from the package license in several of the options in this survey.
What They Give Up#
Speed of delivery. This analysis takes weeks and has to happen before the architecture, which is not how feature work is usually scheduled. The alternative is doing it after, when the answer changes the architecture.
Some of the best models. The highest-accuracy option in the category is research-only. The broadest attribute API belongs to a company on the US Entity List. The most convenient hosted identification service is not sold to customers without a Microsoft account team. The compliance-clean subset is smaller and less capable, and that is the actual state of the category rather than a failure of research.
When the Analysis Changes#
- If the deployment is EU-only or EU-inclusive, the AI Act timeline matters. Article 113 sets general application from 2 August 2026, with Chapters I and II — which contain the Article 5 prohibitions — applying from 2 February 2025, and Article 6(1) and its obligations from 2 August 2027. An amending package moving Annex III high-risk obligations later has been widely reported; this survey could not verify its publication and does not rely on it.
- If users are in Illinois, the private right of action changes the risk calculus regardless of how careful the engineering is.
- If the feature is law enforcement or public-space identification, this survey is not the right document and a library comparison is not the right starting point.
Decision Criteria, Compressed#
Ask three questions before choosing a model: are we storing a template, does data leave the building, and is this verification or identification. The answers eliminate more options than any benchmark does, and they do not change when the model does.
Sources#
- 740 ILCS 14 (BIPA), Illinois General Assembly — read 2026-08-25
- Tex. Bus. & Com. Code § 503.001 (CUBI), as amended by Acts 2025, 89th Leg., R.S., Ch. 1174 (H.B. 149), eff. 2026-01-01 — text read 2026-08-25 from an archived capture of statutes.capitol.texas.gov dated 2025-10-04, because the live site now serves a JavaScript application that returns no statutory text to a non-browser client
- Regulation (EU) 2024/1689, Articles 5, 6 and 113 and Annex III — read 2026-08-25 via the AI Act Explorer reproduction at artificialintelligenceact.eu; EUR-Lex returned an HTTP 202 challenge page to every automated request made for this survey
- GDPR Article 9
- Limited Access features of Face — Microsoft Learn
Use Case: The Embedded Integrator#
Who Needs This#
An engineer building a device rather than an application. A smart doorbell that wakes the main processor when someone is at the door. A retail counter that logs how many people looked at a display. A camera that starts recording when a face appears. A kiosk that dims its screen when nobody is in front of it.
The compute is a Raspberry Pi class board, or smaller. There may be no network. There is a flash budget measured in megabytes and a power budget that decides battery life.
Why They Need It#
This persona needs detection and nothing else, and the distinction carries more weight than it looks like it does.
“Is a face present, and roughly where?” is a question that can be answered in a few hundred kilobytes on a device with no accelerator. “Whose face is this?” needs a model two orders of magnitude larger, a gallery, a threshold, and — as the compliance persona develops — a legal posture. The integrator who accidentally builds the second when they needed the first has taken on a product they did not want.
The engineering constraints are unforgiving in a way that server work is not. Flash is finite and shared with the application. RAM is measured in tens of megabytes. There is no swap. A model that needs a Python interpreter, a TensorFlow runtime and 200 MB of weights is not a candidate at any accuracy.
Requirements#
- Detection only; no embedding, no gallery, no identity
- Model in the hundreds of kilobytes, not the hundreds of megabytes
- Runs on CPU without an accelerator, and preferably from C++ rather than Python
- No network fetch at runtime — the model is vendored into the firmware
- A permissive license, because the model is redistributed inside a physical product the customer owns
- ARM support that does not require building a toolchain from scratch
Which Options Survive#
YuNet, at 227 KB, or 98 KB quantized.
The size is the argument, and the quantization result is what makes it decisive. From opencv_zoo’s own evaluation:
| Model | Easy AP | Medium AP | Hard AP |
|---|---|---|---|
| YuNet | 0.8844 | 0.8656 | 0.7503 |
| YuNet int8 quantized | 0.8810 | 0.8629 | 0.7503 |
Int8 quantization costs 0.0034 AP on the easy subset, 0.0027 on medium, and nothing measurable on hard, while halving the file to 100,416 bytes. A detector under 100 KB that loses nothing on the hard subset is an unusual artifact and it is why YuNet shows up on hardware that has no business running a neural network.
The model file is MIT-licensed with a LICENSE beside it, which is what
this persona needs when the model ships inside a device that will be
resold. It runs through OpenCV’s C++ API, so no Python interpreter is
required on the device.
dlib’s HOG detector is the fallback with a specific virtue: it needs no
model file at all. The weights are base64-serialized into
frontal_face_detector.h, and the header states “License: Boost Software
License”. One dependency, one license, no download, no ONNX runtime, no
vendored artifact to track. On a device where the build is C++ anyway and
the faces are frontal and close, that simplicity is worth something. It
detects only near-frontal faces — five HOG filters, no profile — and it is
slower per frame than YuNet.
OpenCV Haar remains the floor. About 1 MB, no DNN module needed, runs anywhere, and carries an Intel License Agreement rather than OpenCV’s Apache-2.0 — permissive either way, but the attribution notice is a different one, which matters if the device’s license file is generated mechanically.
What Is Eliminated, and Why#
- Anything with a recognition model. InsightFace’s smallest pack is 16 MB and still research-only. SFace is 37 MB. dlib’s ResNet is ~22 MB plus a source build. All of them answer a question this device does not ask.
- MediaPipe. The wheel is 18-38 MB, it pulls in
opencv-contrib-python, and the value it adds — 478 landmarks, blendshapes, a pose matrix — is exactly the value this persona has no use for. There is a C++ path, and it is a Bazel build. retina-faceandmtcnn. Both require TensorFlow. That is the disqualification, before any discussion of model size.- dlib as a Python dependency.
pip install dlibcompiles from source against a 3.3 MB sdist with no wheels, including on ARM. On a cross-compilation target that is a project. dlib as a vendored C++ library is a different and reasonable proposition. - Everything hosted. No network is a hard constraint, and a doorbell that stops working when the WiFi drops is a bad doorbell.
What They Give Up#
Small and distant faces. YuNet’s model card states the trained range explicitly: “This model can detect faces of pixels between around 10x10 to 300x300 due to the training scheme.” A camera looking down a corridor, or one where the subject fills the frame, is outside that range at one end or the other. The mitigation is resolution management — tile or downscale so faces land in range — and it is a real design constraint, not a footnote.
Hard-case accuracy. 0.7503 hard AP means roughly a quarter of the difficult faces in WIDER FACE are missed. For a wake-on-presence trigger that is fine, because the person will still be there next frame. For anything where a single miss is a failure, it is not.
Any path to identity later. This is a feature. If the product later needs to know who is at the door, that is a new product with a new compliance posture, and the integrator should reopen the question rather than bolting an encoder onto a detector.
When This Recommendation Flips#
- If the device has an NPU or a GPU — a Jetson, a phone-class SoC — the budget opens up and MediaPipe’s C++ path or a MobileNet RetinaFace becomes reasonable. The license questions come back with them.
- If the device does need identity, stop and read the compliance persona before choosing a model. An enrolled face template on a device the customer owns raises questions about retention and destruction that several statutes address directly.
- If the only requirement is “someone is there” and faces are incidental, a person detector or a passive infrared sensor may be the better answer, and neither is biometric.
Decision Criteria, Compressed#
Detection only, kilobytes, no network, redistributed inside a device: YuNet int8 at 98 KB, MIT-licensed, through OpenCV’s C++ API. If the build must have zero model artifacts, dlib’s HOG detector ships its weights inside the library under the Boost license.
Use Case: The Inheritor#
Who Needs This#
An engineer who did not choose any of this. They picked up a repository — from a departed colleague, an acquisition, a contractor, or their own past self — and it does face work. The build broke, or a dependency audit flagged something, or someone asked whether it can ship in a new market.
They are not evaluating a category. They are trying to find out what they have.
Why They Need It#
Face code ages badly in a specific way. The API surface stays stable for years while everything underneath it moves: the package stops getting releases, the framework it pins goes two major versions ahead, the model file it downloads is served from a URL somebody stopped paying for, and the license question nobody asked in 2019 becomes a procurement blocker in 2026.
The inheritor’s problem is that none of this announces itself. import face_recognition works. The tests pass. The container built last year.
Nothing is broken until the base image changes.
Three specific inheritances are common enough to name, and this survey’s verification pass found all three still widely in use.
Requirements#
- Establish what is actually installed, including transitive model downloads
- Establish the license of every model file, separately from the package
- Establish whether the upstream project is alive, dormant, or finished
- Get to a supported configuration without re-enrolling anyone if possible
What They Probably Have#
Inheritance 1: face_recognition over dlib.
The most likely one, because it has 56,682 stars and the friendliest API in the category. What the inheritor has:
- A package whose last release was 2020-02-20, with 57 open pull
requests, one merged in the six months to 2026-08-25, and 832 open
issues. The last commit on
master, dated 2026-06-25, is a merge of a contributed link fix. - A hard dependency on
dlib, which has no wheels on PyPI — release 20.0.1 ships a single 3.3 MB sdist — so every clean build compiles C++. - A default path through
shape_predictor_68_face_landmarks.dat, whose own author writes in the model repository’s README that the training dataset’s “license for this dataset excludes commercial use” and that “the trained model therefore can’t be used in a commercial product”.
The build failure and the license finding usually arrive together, because both surface during the same dependency audit.
Inheritance 2: mtcnn or facenet-pytorch.
ipazc/mtcnn has not been committed to since 2024-10-08, and has zero
open pull requests and zero merged — which reads as finished rather than
stalled, since nothing is arriving either. timesler/facenet-pytorch has
not been committed to since 2024-08-02, has 7 pull requests waiting and
0 merged, and its last release was 2024-04-29.
The difference matters. MTCNN has no framework underneath it worth
breaking. facenet-pytorch is pinned against a PyTorch that keeps moving,
which is a countdown someone else is running.
Both bring TensorFlow or PyTorch into a service that may not otherwise need one, which is usually the largest line in the container.
Inheritance 3: insightface from before May 2026.
An older insightface in a requirements.txt means 0.7.3 or earlier —
sdist only, so the build needs a compiler — and it means the code calls
FaceAnalysis(name='buffalo_l'), which downloads a 326 MB model pack that
InsightFace states is “for non-commercial research purposes only”,
including the auto-downloaded ones.
If the product is commercial, that is the finding, and it was there from the first run.
What Survives, and What to Do#
The triage, in order.
- Find the model files, not the packages. Look in
~/.insightface/models/,~/.deepface/weights/, and wherever the code pointsdlib.shape_predictoranddlib.face_recognition_model_v1. The package license is not the answer to the license question. - Establish whether templates are stored. If enrolled embeddings exist in a database, the encoder cannot be swapped without re-enrolling everyone, and that constrains everything below.
- Establish whether the upstream is alive. Last commit, last release, open-versus-merged pull requests over six months. This survey’s S2 files have the figures as of 2026-08-25; re-run them, because they move.
The migrations that are cheap.
- Detector only, from MTCNN or Haar to YuNet.
cv2.FaceDetectorYNis in the OpenCV already present, needs a 227 KB MIT-licensed file, emits the same five landmarks MTCNN does, and removes a TensorFlow dependency. This is usually an afternoon and it is the single highest-value change available to this persona. shape_predictor_68_face_landmarkstoshape_predictor_5_face_landmarks, where the 68 points were only ever feeding an alignment step. Same library, no restriction note in the model README, much smaller file. If the 68 points feed a visualization or a geometry feature, this is not available.
The migration that is not cheap.
- Changing the encoder. Every stored template is in the old model’s embedding space and none of them transfer. Changing from dlib’s ResNet to SFace, or from ArcFace to anything, means every enrolled person enrolls again. If the enrolled population is customers, that is a product event with a communications plan, not a refactor.
The mitigation for next time is the one the research-lab persona also lands on: keep the encoder behind an interface, so this is a decision rather than a rewrite.
What They Give Up#
The option of doing nothing. A dormant dependency is stable until the base image, the Python version, or the framework moves, and then it is not. The inheritor can defer, but the deferral has an owner and a date somebody else set.
Bug fixes upstream. On face_recognition and facenet-pytorch, a
patch is a fork. The forks exist — ageitgey/face_recognition has 13,699
of them — and picking one is picking a maintainer with no commitment to
you.
Accuracy, sometimes, in exchange for maintenance. Moving from a dlib ResNet encoder to SFace trades a 0.993833 LFW figure for a 0.9940 one on a benchmark that separates neither, with no comparable data on the harder sets. The reason to move is the build and the license, not the accuracy, and the inheritor should say so when asked.
When This Changes#
- If the code is not commercial — an internal tool, a research artifact — the license findings mostly evaporate and the question narrows to whether the build still works.
- If the code is unmaintained but working and frozen, and the deployment is frozen too, doing nothing is a defensible position with a documented expiry rather than an oversight.
- If templates are stored and the license is the problem, there is no cheap path, and the decision belongs with whoever owns the product.
Decision Criteria, Compressed#
Audit the model files, not the packages. Swap the detector first because it is cheap and removes the most dependency weight. Do not swap the encoder unless the license forces it, because every stored template is in its embedding space.
Use Case: The Product Engineer Shipping Face Matching#
Who Needs This#
A senior engineer at a company with a product and paying customers. The feature is “group photos by person” in a media app, or “unlock this workstation with your face”, or “match the person at the counter against their loyalty profile”. It is a shipping feature in a product that is sold.
They are not a computer vision specialist. They have read that InsightFace
is the most accurate thing available and were about to pip install insightface.
Why They Need It#
Face matching in a commercial product is a chain of small decisions that each look reversible and are not:
- The encoder determines the embedding space. Change encoders later and every enrolled template is worthless; everyone re-enrolls.
- The threshold determines the false-match and false-reject rates, and it has to be chosen against the actual population, not copied from a tutorial.
- The license determines whether any of this can ship at all, and it is the one that gets discovered last — usually by legal, during a procurement review, after the feature is built.
The third one is what this persona is here for. The engineer’s real constraint is not accuracy. It is that they work at a company that cannot have a research-only model in a shipped binary, and cannot spend a quarter negotiating a model license with a research group for a feature that is one line item on a roadmap.
Requirements#
- Detection and recognition, both
- Weights whose license permits commercial redistribution, verifiable from a file in the repository, not from a README’s tone
- Runs on CPU on ordinary server hardware; a GPU should be an optimization, not a requirement
- Small enough to ship in a container without a model-download step at runtime, or with a vendorable one
- Maintained: a security advisory needs somewhere to land
- Deployable on the platforms the product already targets
Which Options Survive#
OpenCV’s YuNet + SFace pair is the answer, and it is already installed.
cv2.FaceDetectorYN runs YuNet: 227 KB, MIT-licensed at the model file,
0.7503 hard AP on WIDER FACE validation by opencv_zoo’s own evaluation, and
it emits the five landmarks that the recognizer’s alignment step wants.
cv2.FaceRecognizerSF runs SFace: 37 MB, Apache-2.0 at the model file,
0.9940 on opencv_zoo’s evaluation, with cosine and L2 matching in the API.
Both model files live in opencv/opencv_zoo with a LICENSE file
alongside them — MIT for YuNet, Apache-2.0 for SFace — which is what
“verifiable from a file” means. Both are callable from the
opencv-python the product almost certainly already depends on. The
combined weight footprint is under 38 MB and both files can be vendored
into the image.
OpenCV itself is the healthiest project in the survey: 600 pull requests merged in six months against 208 open, a commit on the day of this pass, an Apache-2.0 license and a foundation behind it.
The runner-up is dlib’s conservative configuration — HOG detector (Boost-licensed, compiled into the library), 5-point shape predictor (no dataset restriction stated), ResNet encoder at 0.993833 LFW. It is a defensible stack and it is more mature than SFace. It costs a source build on every platform, which is a real tax in CI, and it requires the engineer to know that the 68-point predictor everyone uses is the one to avoid.
What Is Eliminated, and Why#
- InsightFace. Highest accuracy in the category on every benchmark that
separates models. The project states in three separate documents that all
its models, including the ones the library downloads automatically, are
“available for non-commercial research purposes only”. Since 2025-11-24
the README names an address to email about licensing
buffalo_l. That is a path, and it is a quarter of somebody’s time and a contract. For most product teams the answer is no; for a team where face matching is the product, it is a conversation worth having. - The
retina-facepip package. MIT code, and its README says its pre-trained weights are the same weights as InsightFace’s. Three MIT re-hostings do not relicense an upstream artifact, and nothing in the chain grants anything about the weights. Unresolved is not the same as permitted. - dlib’s 68-point predictor, and anything that loads it —
face_recognitionby default, several DeepFace paths. The model’s author writes that it “can’t be used in a commercial product”. An engineer who ships it has shipped a documented restriction. face_recognition. Last release 2020-02-20, 57 open pull requests, one merged in six months, 832 open issues. Whatever its API virtues, it is not somewhere a security advisory can land.- DeepFace as a default. Not eliminated — it is maintained, tidy and
useful — but it is a configuration rather than a choice. Its README
states that “License types will be inherited when you intend to utilize
those models”, and its backends span MIT, Apache-2.0, AGPL-3.0
(
yolov8, via Ultralytics) and research-only (Buffalo_L). A product team using DeepFace must pindetector_backendandmodel_nameand treat those two strings as license-bearing configuration, reviewed like a dependency.
What They Give Up#
Accuracy on the hard benchmarks. SFace publishes one number, 0.9940 on a benchmark every modern model saturates, and no IJB-C, no MR-ALL, no demographic breakdown. InsightFace’s buffalo_l publishes IJB-C(E4) 97.25 and MR-ALL 91.25 and is presumably ahead. How far ahead is unmeasured publicly, because nobody has run SFace on those sets and published it. The engineer is trading a known-good license for an unmeasured accuracy gap, and that trade should be stated to whoever asks rather than hidden.
The demographic question stays open. InsightFace publishes a spread of almost twenty points between its best and worst demographic subset on buffalo_l. SFace and dlib publish nothing comparable. Silence is not parity. If the product will be used by a population that is not well-represented in the training data, the engineer has to measure it themselves, on their own population — which is also the only measurement that would have been decision-relevant anyway.
Attribute analysis. No age, gender or emotion from this stack. That is arguably a feature; see the compliance persona and the EU AI Act’s treatment of emotion inference and biometric categorization.
When This Recommendation Flips#
- If face matching is the product, not a feature, the accuracy gap becomes the business and the InsightFace licensing conversation is worth having. Budget a contract, not an afternoon.
- If the gallery is large and the matching is 1:N at scale, the encoder matters less than the index. Once the embedding is computed, the search is a nearest-neighbor problem and belongs to a different survey (1.009).
- If the team has no ML capacity at all and volume is low, Amazon Rekognition at $1.00 per 1,000 images removes the model question entirely, at the cost of sending face images to a third party — which the compliance persona will have opinions about.
- If the deployment is a browser, SFace through ONNX Runtime Web is workable and MediaPipe cannot help, because MediaPipe has no recognition model.
Decision Criteria, Compressed#
Commercial product, needs detection and recognition, cannot negotiate a model license:
cv2.FaceDetectorYN+cv2.FaceRecognizerSF. It is in the library you already import, it is 38 MB, and the license files are in the repository next to the weights.
Use Case: The Research Lab#
Who Needs This#
A graduate student or research engineer at a university or a corporate research group. The output is a paper, a benchmark submission, or an internal study. Nothing is being sold and nothing is being shipped to end users.
They need to be at or near the state of the art, they need to be able to compare against what everyone else compares against, and they need reviewers to be able to reproduce the setup.
Why They Need It#
Research has an inverted constraint set relative to every other persona in this survey. The non-commercial restriction that eliminates InsightFace for a product team is satisfied by construction here — “non-commercial research purposes only” describes the use exactly. What eliminates options for this persona is the opposite: a model nobody else uses is a model whose numbers nobody can situate.
Reproducibility is the second constraint and it is the one people underestimate. A paper that reports results against “the default face detector” is unreproducible three years later when the default has moved. Pinning a model file by name, version and hash matters more than which model it is.
Requirements#
- State-of-the-art recognition, so results are comparable to current work
- Published numbers on the benchmarks the field uses — IJB-B, IJB-C, CFP-FP, AgeDB-30, and the MFR-style demographic splits, not only LFW
- Training code, not only inference, if the work involves fine-tuning
- Model zoo breadth: multiple backbones, multiple training sets
- Pinnable artifacts
Which Options Survive#
InsightFace, without much competition.
The model zoo is the reason. It publishes packs and standalone models across backbones (MobileFaceNet, ResNet50, ResNet100) and training sets (WebFace600K, Glint360K, MS1MV2, MS1MV3), with a results table that includes the harder benchmarks:
| Name | MR-ALL | African | Caucasian | South Asian | East Asian | LFW | CFP-FP | AgeDB-30 | IJB-C(E4) |
|---|---|---|---|---|---|---|---|---|---|
| buffalo_l | 91.25 | 90.29 | 94.70 | 93.16 | 74.96 | 99.83 | 99.33 | 98.23 | 97.25 |
| buffalo_s | 71.87 | 69.45 | 80.45 | 73.39 | 51.03 | 99.70 | 98.00 | 96.58 | 95.02 |
Nothing else in this survey publishes a table like that. SFace publishes one number. dlib publishes one number. The demographic columns in particular are a research resource — a lab studying fairness in face recognition has a vendor-published baseline to work against, which is rare.
The repository also carries the training code for ArcFace, RetinaFace and SCRFD, so fine-tuning and ablation are supported rather than reverse engineered. And the lineage is coherent: RetinaFace and ArcFace come from the same group, and the RetinaFace paper’s own code link points into this repository.
The 1.0 release helps this persona specifically. InsightFace’s
Evaluation Studio, added 2026-05-23, reports “the best cosine threshold
accuracy, the selected threshold, positive and negative pair counts, and
TAR@FAR for 1e-6, 1e-5, 1e-4, and 1e-3 with corresponding
thresholds” over a local dataset. That is the correct characterization of a
matcher and it is now a GUI rather than a script somebody has to write.
dlib is the secondary pick for a specific kind of work: it is a complete, readable, C++ implementation with a metric-learning loss documented in plain language, and it is a good teaching and ablation substrate. It is not competitive on current benchmarks.
What Is Eliminated, and Why#
- MediaPipe. No recognition model, and its geometry models are shipped
as opaque
.taskbundles rather than research artifacts. It is a deployment technology, not a research one. - OpenCV YuNet + SFace. SFace’s published evaluation is a single saturated benchmark. A paper reporting SFace numbers cannot be situated against the current literature, because the current literature does not report SFace.
- The hosted APIs. Unreproducible by construction: the model changes under you, the vendor publishes no benchmark numbers, and a result obtained against “Rekognition in August 2026” cannot be re-obtained.
face_recognitionandfacenet-pytorchas primary tools. Both are dormant.facenet-pytorch’s VGGFace2 and CASIA-WebFace embeddings remain a legitimate historical baseline to compare against, and a paper citing them should pin the version — 2.6.0, 2024-04-29 — because nobody is maintaining it.
What They Give Up#
Any path to production. This is the trade, and it should be made with
open eyes. A lab that builds its pipeline on InsightFace and then spins out
a company, or hands the work to a product team, hands over a codebase whose
central model is licensed for research only. The conversation with
[email protected] happens at the worst possible moment
— after the architecture is set.
The mitigation is cheap and should be standard: keep the encoder behind an interface from day one. If swapping InsightFace for SFace is a one-file change, the license conversation is a decision rather than a rewrite. If ArcFace embeddings are threaded through the codebase, it is a rewrite.
Maintenance responsiveness. InsightFace has 1,266 open issues and merged 3 pull requests in the six months to 2026-08-25. The models improve; the issue tracker does not move. A lab should expect to read the source.
A license file. deepinsight/insightface has no LICENSE at the
repository root — GitHub reports the license as null and PyPI reports none,
after the 1.0.1 changelog recorded removing “the PyPI package metadata
license classifier field while keeping the README license guidance”. The
terms are prose in three READMEs. For a lab this is a citation and
compliance-paperwork annoyance; for the product team downstream it is worse.
When This Recommendation Flips#
- If the work is about detection rather than recognition, the field’s comparison point is WIDER FACE, and RetinaFace, SCRFD and YuNet are all legitimate baselines with published numbers. YuNet’s opencv_zoo evaluation is reproducible with the tooling in the repository, which is more than most.
- If the work will be commercialized, run the encoder-behind-an- interface discipline from the first commit and validate SFace as the fallback early, so the accuracy delta is a known number rather than a surprise.
- If the work is about demographic fairness, InsightFace’s published splits are a starting point and not an endpoint; the numbers are the vendor’s own and the evaluation protocol is theirs.
Decision Criteria, Compressed#
Research use, accuracy is the objective, non-commercial is satisfied: InsightFace, and keep the encoder behind an interface so the license conversation stays a decision rather than a rewrite.
S4: Strategic
Accuracy Claims: Where the Numbers Come From#
Face recognition is a field with a strong benchmark culture and an unusually weak independent-verification culture at the open-source end. Almost every number in this survey — including every number in S1 — was measured by the organization that published the model, on a benchmark of its choosing.
That does not make the numbers wrong. It makes them a specific kind of evidence, and a survey that tabulates them without saying so is misleading its readers.
Everything below was checked 2026-08-25.
Who Measured What#
| Figure | Model | Measured by | Where published |
|---|---|---|---|
| 91.4 AP, WIDER FACE hard test | RetinaFace (paper config) | The paper’s authors | arXiv:1905.00641 abstract |
| 80.99 hard val | RetinaFace MobileNet-0.25 | The re-implementation’s author | biubug6/Pytorch_Retinaface repo description |
| 0.8844 / 0.8656 / 0.7503 WIDER FACE val | YuNet | opencv_zoo’s own tools/eval | opencv_zoo model card |
| 0.834 / 0.824 / 0.708 WIDER FACE val | YuNet | The model card’s headline text (a different evaluation from the table below it) | opencv_zoo model card |
| 99.83 LFW, 97.25 IJB-C(E4), 91.25 MR-ALL | InsightFace buffalo_l | InsightFace | Model zoo README |
| 0.9940 “Accuracy” | SFace | opencv_zoo’s tools/eval | opencv_zoo model card |
| 0.993833 ± 0.00272732 LFW | dlib ResNet | Davis King | dlib-models README |
| 0.9895 LFW | Taguchi dlib variant | Taguchi (third party to dlib) | dlib-models README, citing the Taguchi repo |
| 96.1 LFW at threshold 0.55 | dlib BAREL DenseNet | The BAREL project | dlib-models README |
| 2.94 ms CPU / 7.41 ms GPU, Pixel 6 | BlazeFace short-range | MediaPipe Face Detector docs | |
| 89.59 TAR at FAR=1e-6, IJB-C | RetinaFace + ArcFace | The paper’s authors | arXiv:1905.00641 abstract |
One line in that table is third-party: the Taguchi model’s figure, and it is third-party to dlib rather than to Taguchi. Everything else is a self-report.
Four Ways These Numbers Mislead#
1. LFW cannot separate modern models#
Three of the recognition options here report on LFW: InsightFace buffalo_l at 99.83, SFace at 0.9940, dlib at 0.993833. That is a spread of 0.45 percentage points across three models with materially different architectures, training sets and sizes.
LFW is a 2007 verification benchmark over 6,000 curated pairs. It has been saturated for years. Within InsightFace’s own family, buffalo_l scores 99.83 and buffalo_s scores 99.70 — thirteen hundredths apart — while on MR-ALL the same two models score 91.25 and 71.87, nearly twenty points apart.
A benchmark that cannot distinguish two models that differ by twenty points on a harder set is not measuring what a chooser needs. S1 leads its recognition comparison with LFW and ranks three models by it. The ranking is inside the noise.
2. WIDER FACE numbers are not one number#
S1’s master table puts “RetinaFace 91.4% (WIDER FACE hard)” and “YuNet 0.708 hard” — where it mentions YuNet at all — as if they were comparable. Three separate qualifications apply:
- Test set versus validation set. RetinaFace’s 91.4 is the test set. YuNet’s figures are validation. Different sets.
- Which configuration. RetinaFace’s 91.4 is the paper’s best configuration with a heavy backbone and the self-supervised mesh decoder branch. The MobileNet-0.25 variant that people actually deploy scores 80.99 hard val, by the re-implementation’s own repository description. An eleven-point gap between headline and deployment.
- Which evaluation harness. YuNet’s own model card carries two
different sets of WIDER FACE validation numbers — 0.834/0.824/0.708 in
the prose and 0.8844/0.8656/0.7503 in the table produced by
opencv_zoo/tools/eval. Same model, same benchmark, different harness, a four-point spread.
The comparable pair in this category is YuNet’s 0.7503 hard val against RetinaFace MobileNet’s 80.99 hard val: same set, same subset, both self-reported. About five points, for roughly seven times the file size and an unresolved weights license.
3. A single FPS number is not a measurement#
S1’s speed table gives eight projects an FPS figure — “OpenCV Haar: 30+ FPS”, “MTCNN: 5-15 FPS”, “Dlib CNN: 1-3 FPS”, “MediaPipe: 60-100 FPS” — with no cited source for any of them.
Detector latency depends on input resolution, thread count, backend (BLAS, OpenVINO, CUDA, CoreML), whether NMS runs on CPU, and — for cascade detectors like MTCNN and Haar — on how many candidates survive each stage, which depends on the image. A single number without those is not reproducible and cannot be compared across projects.
The only vendor-published latency figure in the whole category is Google’s: 2.94 ms CPU / 7.41 ms GPU for BlazeFace short-range on a Pixel 6. Note that GPU is slower, which is what you expect for a 128 x 128 model where transfer dominates — and which quietly contradicts the intuition that adding a GPU speeds up face detection.
Every FPS number in S1 is recorded in S2 as not verified. Running a cross-project benchmark on comparable hardware would settle it, and that is a build rather than a survey.
4. The demographic gap is published exactly once#
InsightFace’s model zoo publishes this alongside the LFW figure everyone quotes:
| Name | MR-ALL | African | Caucasian | South Asian | East Asian |
|---|---|---|---|---|---|
| buffalo_l | 91.25 | 90.29 | 94.70 | 93.16 | 74.96 |
| buffalo_s | 71.87 | 69.45 | 80.45 | 73.39 | 51.03 |
A spread of 19.74 points between best and worst demographic subset on buffalo_l, and 29.42 points on buffalo_s. Published by the model’s own authors, on the same table as the 99.83.
Two readings, and both belong in a survey.
First, this is a fact about the model that changes deployments. A 1:N identification system whose per-demographic accuracy varies by twenty points will not fail evenly across a user population, and a threshold tuned on one population is mistuned for another.
Second, InsightFace is the only project in this survey that publishes this at all. SFace publishes one aggregate number. dlib publishes one aggregate number. MediaPipe publishes none. The hosted APIs publish none. Their silence is not evidence of parity — it is evidence that nobody measured, or nobody published.
Crediting InsightFace for the disclosure and penalizing it for the number would be the wrong reading of the same table. The disclosure is the better behavior. The number is a constraint on where the model belongs.
The existence of taguchi_face_recognition_resnet_model_v1.dat in dlib’s
own model repository — trained on 6.5 million faces of which “Approximately
47% of this is facial data of Japanese people”, offered as a drop-in
replacement for the default — is the same phenomenon showing up as a
community workaround rather than as a published measurement.
Where Independent Numbers Exist#
NIST runs the only large-scale independent evaluation of face algorithms. The program is now FRTE/FATE, which NIST describes as a restructuring of the former FRVT: “tracks that involve the processing and analysis of images will run under the FATE activity, and tracks that pertain to identity verification will run under FRTE”. FRTE covers 1:1 verification, 1:N identification, Face in Video Evaluation and a twins demonstration; FATE covers morph detection, quality assessment, presentation attack detection and age estimation. NIST states the program “is an ongoing activity and runs continuously”, with participants able to submit roughly every four calendar months.
None of the open-source models in this survey are FRTE participants. Submission requires packaging an algorithm to NIST’s API and a commitment most research groups and single maintainers do not make. So for InsightFace, SFace, dlib, YuNet, MTCNN and RetinaFace there is no independent number at all, and the ones in this survey are the only ones there are.
NIST’s reports are the place to look for numbers on commercial submissions, and for the demographic-differentials methodology that FATE’s own tracks formalized. A reader whose deployment turns on accuracy at a specified false-match rate should read NIST’s reports rather than any table in this survey.
The Benchmark a Reader Should Actually Run#
The measurement that decides a deployment is not on any of these tables. It is:
On your own population, at your own operating point, what is the false match rate and the false non-match rate?
That is a different question from “what does this model score on LFW”, and every element of it is local: the images your camera produces, the demographics of your users, the size of your gallery (false matches scale with gallery size in 1:N), and the threshold you chose.
InsightFace’s Evaluation Studio, added in 1.0, produces exactly this shape
of output over a local dataset — “the best cosine threshold accuracy, the
selected threshold, positive and negative pair counts, and TAR@FAR for
1e-6, 1e-5, 1e-4, and 1e-3 with corresponding thresholds”. Whatever
one concludes about the licensing, that is the right report, and it is the
report a team should be producing for whichever model they choose.
The threshold trap, specifically. dlib’s recognition README explains
that its model was trained with “a structured metric loss that tries to
project all the identities into non-overlapping balls of radius 0.6”, which
is where the 0.6 in every dlib face-matching tutorial comes from. It is a
training hyperparameter. Copying it as an operating threshold means
accepting whatever false-match rate it happens to produce on your gallery,
which nobody has measured. The BAREL variant in the same repository uses
0.55, and the number changes with the model.
Sources#
- arXiv:1905.00641 — RetinaFace abstract
- biubug6/Pytorch_Retinaface — repository description, MobileNet-0.25 hard val figure
- opencv_zoo YuNet model card and SFace model card
- opencv_zoo tools/eval — supported datasets
- InsightFace Model Zoo — accuracy and demographic tables
- davisking/dlib-models README — LFW figures, loss description, Taguchi and BAREL variants
- MediaPipe Face Detector guide — Pixel 6 latency
- NIST FRTE/FATE — program description and track structure
- InsightFace commercial evaluation doc — TAR@FAR reporting
S4: Strategic Discovery — Approach#
Date: 2026-08-25.
What Makes This Category’s Strategy Different#
For most software categories, S4 asks the durability questions: who funds this, who maintains it, what happens if they stop, how hard is it to leave. Those questions apply here and this pass answers them.
But two things are load-bearing in face software that are footnotes almost everywhere else, and getting them wrong costs more than any of the usual durability risks:
1. Licensing is decided at the weights, and the weights disagree with the package.
This is not a generic “check your licenses” reminder. In this specific
category, the highest-accuracy recognition model is research-only, the
most-reproduced landmark model carries an author’s note saying it cannot be
used in a commercial product, and the pip package that redistributes the
best-known detector’s weights is MIT over an upstream that is not. Three of
the most-installed artifacts in the category have a license position that
differs from the badge on their package. licensing-chain.md resolves each
one against its own repository and model card.
2. This is biometric software, and the regulatory surface is an engineering input.
A team choosing a JSON parser does not need to know what Illinois thinks about it. A team choosing a face encoder does. Notice, consent, retention, destruction, private rights of action, and the distinction between verification and remote identification are all settled by statute in several places where the software will run, and they constrain the architecture — where templates live, how long they live, whether the feature can exist at all — before any model is chosen.
regulatory-surface.md reports what three regimes say and where each was
read. It does not give legal advice, does not predict enforcement, and
does not tell any reader whether they are compliant. The useful framing,
borrowed from how another survey in this library handles the AGPL, is
where this question arises: which architectural choices put you inside a
rule’s scope and which keep you outside it. That is knowable from the text
of the rules. Everything past it needs counsel.
The Other Three Questions#
maintenance-and-durability.md applies ADDING-RESEARCH’s stall test to
every project in the survey and finds one result the test was not designed
for — a project with zero merges and zero open pull requests, which is
finished rather than stalled — and one false positive, a Google project
whose community pull requests never merge by design.
accuracy-claims.md traces where every number in this category comes from.
The short answer is that essentially all of them are self-reported by the
organization that published the model, that the benchmark everyone quotes
cannot separate the models, and that the one project publishing demographic
breakdowns publishes unflattering ones — which counts in its favor on
transparency.
exit-and-portability.md asks what leaving costs. The answer differs by
an order of magnitude between the three jobs: detectors are interchangeable
in an afternoon, landmark consumers are a rewrite, and encoders cannot be
changed at all once templates are stored without re-enrolling every person.
Method#
Same discipline as S2. Every license term was read from the project’s own LICENSE file, README or model card, fetched from the default branch on 2026-08-25. Every dataset restriction was read at the institution that publishes the dataset. Every statutory quotation was read from the government’s own publication of the text, with one exception that is disclosed at the point of use: the Texas statute’s live site now serves a JavaScript application that returns no statutory text to a non-browser client, so that text was read from an archived capture and the capture date is given.
The EU AI Act text was read from a third-party reproduction because EUR-Lex returned an HTTP 202 challenge page to every automated request made for this survey. That is disclosed at the point of use as well, and it is a weaker sourcing than the rest of this pass. A refresh should re-read those articles from the Official Journal.
Structure of This Pass#
| File | Question |
|---|---|
licensing-chain.md | What am I allowed to ship, per model, with the evidence |
regulatory-surface.md | Where does the biometric question arise, and what do the rules say |
maintenance-and-durability.md | Which of these will still be here, and which already stopped |
accuracy-claims.md | Where do the numbers come from, and what can they support |
exit-and-portability.md | What does it cost to change my mind |
recommendation.md | Strategic paths |
Exit and Portability: What Changing Your Mind Costs#
Lock-in in this category is not where people expect it. The framework is portable, the container is portable, the API surface is small. The thing that traps you is a number: a 128- or 512-dimensional vector that means nothing outside the model that produced it.
Everything below was checked 2026-08-25.
The Three Exit Costs Are Not Comparable#
| What you are replacing | Cost | Why |
|---|---|---|
| A detector | An afternoon | Bounding boxes are bounding boxes. Every detector produces the same shape |
| A landmark consumer | A rewrite of the consumer | 5, 68, 106, 468 and 478 points are five different coordinate conventions |
| An encoder, with templates stored | Every enrolled person re-enrolls | Embeddings are model-specific and do not transfer |
That spread — an afternoon, a sprint, a product event — is the single most useful thing to know before choosing anything in this category.
Detectors Are Nearly Free to Replace#
A face detector’s output is a list of boxes and, usually, five landmarks. YuNet, MTCNN, RetinaFace, BlazeFace and dlib’s HOG detector all produce that, in slightly different array layouts and coordinate conventions. Swapping one for another is an adapter function and a threshold retune.
The reason this matters: the highest-value change available to most
readers of this survey is a detector swap, and it is cheap. Moving from
MTCNN or a Haar cascade to cv2.FaceDetectorYN removes a TensorFlow
dependency or a 2001 model, gains accuracy, drops to a 227 KB MIT-licensed
file, and is an afternoon’s work in a codebase that is already using
OpenCV.
Two things that do change and need checking after a swap:
- Box conventions. Some detectors return
(x, y, w, h), some(x1, y1, x2, y2), and the tightness of the crop differs. A recognition pipeline tuned on one detector’s crops will drift on another’s. - Alignment. dlib’s model README warns about this directly: the 68-point predictor “is designed for use with dlib’s HOG face detector … It won’t work as well when used with a face detector that produces differently aligned boxes, such as the CNN based mmod_human_face_detector.dat face detector.” The same caution applies to any detector-plus-predictor pairing. Silent degradation, no error.
Landmarks Are a Convention, Not a Format#
There is no interchange standard for facial landmarks. What each option produces:
| Option | Points | Convention |
|---|---|---|
| YuNet, MTCNN, RetinaFace | 5 | Eye centers, nose tip, mouth corners |
| dlib | 68 | The iBUG 300-W / Multi-PIE 68-point mark-up |
| dlib | 5 | dlib’s own, for alignment |
| InsightFace | 106 (2D), 68 (3D) | InsightFace’s own |
| MediaPipe (Tasks) | 478 | Canonical face mesh topology |
| MediaPipe (legacy) | 468, or 478 refined | Same topology, iris points optional |
Code that consumes landmarks is written against one of these conventions
by index — landmarks[36:42] is the left eye in the 68-point mark-up and
nothing in particular in any other. Changing conventions means rewriting
every consumer.
The mitigation is to consume as few points as the job needs. A pipeline that uses landmarks only to align a face before encoding needs five, and five points are the one convention every detector in the category agrees on. A pipeline that renders geometry needs the mesh and is coupled to MediaPipe, which is a reasonable coupling because nothing else produces a mesh.
The licensing overlay: the 68-point convention is also the one with the commercial-use note on its most common model. A codebase that consumes 68 points by index is coupled both to a convention and to a restricted model.
Encoders Are Where the Lock-In Lives#
An embedding is a point in a space defined by the weights that produced it. There is no conversion, no adapter, no re-projection. dlib’s 128-d vector and InsightFace’s 512-d vector are not different formats of the same thing; they are different things.
So: if you have stored templates, changing the encoder means every enrolled person enrolls again.
The cost scales with who is enrolled:
| Enrolled population | Re-enrollment cost |
|---|---|
| Internal, a few dozen | An afternoon and an email |
| Employees, a few thousand | A scheduled campaign, a fallback path, a help desk |
| Customers | A product event — communications, consent, churn, support load |
If the original images were retained, re-enrollment can be done in batch without contacting anyone — and whether the original images should have been retained is a question the compliance material has views about, since retaining source images to preserve the option of re-encoding is in tension with destroying data when the purpose is satisfied.
This is why the interface discipline matters and why two separate S3
personas arrive at it independently. Keep the encoder behind a narrow
interface — encode(aligned_face) -> vector, compare(a, b) -> score —
from the first commit. Then a license change, an accuracy upgrade or a
project going dormant is a decision with a migration plan, rather than a
rewrite with a re-enrollment attached.
Runtime Portability Is Good, and It Is Not the Problem#
The runtime picture in this category is unusually healthy and is worth saying plainly so it does not get confused with the encoder problem.
ONNX is the common currency. InsightFace ships ONNX and runs on ONNX
Runtime (1.29.0, uploaded 2026-08-17 — a live, fast-moving project with
90 million monthly downloads). opencv_zoo ships ONNX. Models can be moved
between CPU, CUDA, CoreML, DirectML and the browser through ONNX Runtime
Web without touching the model.
MediaPipe’s .task bundles are the exception, and the exception buys
something: they run through Google’s own Android, iOS and Web bindings,
which is why MediaPipe holds the on-device geometry niche. The bundle is
not portable outside MediaPipe. Since MediaPipe produces geometry rather
than templates, that coupling is recoverable — no stored artifact depends
on it.
dlib’s .dat format is dlib-only, and dlib is C++ with no wheels. That
is a build constraint rather than a portability constraint: the model runs
wherever you can compile dlib, which is most places, expensively.
Hosted API Exit#
Leaving a hosted service has all of the encoder problem plus one addition: you cannot export the templates.
Amazon Rekognition Collections hold face vectors server-side. Face++ FaceSets and Azure PersonGroups are the same arrangement. The APIs are built around indexing and searching, not around exporting the vectors, and the vectors would be useless outside the vendor’s model anyway.
So exiting a hosted face service means re-enrolling from source images, if you kept them, or from the people, if you did not. The migration is the same shape as an encoder change and it is not cheaper for having been someone else’s model.
The corollary for the direction people usually plan: “start with the cloud API, move to self-hosted at scale” is a migration with a re-enrollment attached, and S1 presents it as a cost-optimization step with a break-even point. The break-even calculation is the easy part.
The Exit Checklist#
For any face system being designed today:
- Put the encoder behind an interface.
encode()andcompare(). Nothing else in the codebase should know which model is loaded. - Store the model identity alongside every template. Model name, version, file hash. Then a mixed-generation gallery is detectable instead of silently wrong.
- Consume the fewest landmark points the job needs. Five if you are aligning; the mesh only if you are rendering.
- Record the threshold and how it was chosen, with the population it was chosen against. A threshold copied from a tutorial is an unmeasured false-match rate.
- Decide the source-image retention question explicitly. Keeping them makes re-enrollment free and creates a retention obligation. Deleting them does the reverse. Either is defensible; drifting into one is not.
- Pin model files by hash, not by package version. The package can change what it downloads.
Sources#
- davisking/dlib-models README — the detector/predictor alignment warning
- onnxruntime on PyPI — 1.29.0, 2026-08-17
- MediaPipe Face Landmarker guide — model bundle structure
- Amazon Rekognition pricing — face vector storage billing, which is where Collections are visible as a stored artifact
- Landmark conventions from each project’s own documentation, as cited in
../S2-comprehensive/feature-comparison.md
The Licensing Chain: What You Are Allowed to Ship#
All terms below were read on 2026-08-25 from the source named beside them. Where a term is quoted, it is quoted verbatim from that source. Nothing here is legal advice; it is a record of what each project publishes about its own artifacts, and of where two projects publish different things about the same artifact.
The Rule That Governs This Category#
package license ≠ weights license ≠ training data licenseEach link is granted separately by a different party. A permissive package license says nothing about the model file the package downloads, and a model author’s own grant does not reach past a restriction in the dataset the model was trained on.
Every trap in this category is an instance of that rule, and every one of them is documented by the project itself. None of this is hidden. It is missed because dependency tooling reads the first link.
The Table#
| Artifact | Code license | Weights license | Training data | Commercial? |
|---|---|---|---|---|
| MediaPipe BlazeFace / FaceMesh-V2 / Blendshape | Apache-2.0 | Google model cards; no research-only restriction found | Not published per-model | Yes |
YuNet (cv2.FaceDetectorYN) | Apache-2.0 (OpenCV) | MIT (LICENSE beside the model) | WIDER FACE, via libfacedetection.train (BSD-3-Clause) | Yes |
SFace (cv2.FaceRecognizerSF) | Apache-2.0 (OpenCV) | Apache-2.0 (LICENSE beside the model) | Not published on the card | Yes |
| OpenCV Haar cascade | Apache-2.0 (OpenCV) | Intel License Agreement in the XML header | — (2001 AdaBoost training) | Yes, under a different notice |
| dlib HOG detector | BSL-1.0 | BSL-1.0 — serialized into the header | Davis King’s own | Yes |
| dlib 5-point predictor | BSL-1.0 | Public domain per author (repo CC0-1.0) | dlib 5-point dataset, 7,198 faces, annotated by King | Author grants it |
| dlib 68-point predictor | BSL-1.0 | Public domain per author, with a countervailing note | iBUG 300-W | No — author states it “can’t be used in a commercial product” |
| dlib 68-point GTX | BSL-1.0 | Same | iBUG 300-W | Same restriction |
| dlib recognition ResNet | BSL-1.0 | Public domain per author | face scrub, VGG dataset, images scraped from the internet | Author grants it; constituent datasets have their own terms |
| dlib CNN detector | BSL-1.0 | Public domain per author | ImageNet, AFLW, Pascal VOC, VGG, WIDER, face scrub | Author grants it; same caveat |
| InsightFace code | MIT | — | — | Yes, no limitation |
| InsightFace all models (buffalo_*, antelopev2, everything in the zoo) | MIT | Non-commercial research purposes only | WebFace600K, Glint360K, MS1MV2, MS1MV3 | No — licensing by email since 2025-11-24 |
retina-face pip package | MIT | Unresolved — same weights as InsightFace’s RetinaFace, per its own README | — | Unresolved |
mtcnn | MIT | Bundled with the package | WIDER FACE / CelebA (per the 2016 paper) | Package grants MIT |
face_recognition | MIT | Whatever dlib model it loads | dlib’s chain | Depends — default path is the 68-point model |
| DeepFace | MIT | Whatever backend you name | Varies | Depends on two string arguments |
facenet-pytorch | MIT | Bundled | VGGFace2, CASIA-WebFace | Datasets have their own terms |
| Ultralytics YOLO (a DeepFace detector backend) | AGPL-3.0 | AGPL-3.0 | — | Copyleft, network-triggered |
Case 1: dlib’s 68-Point Predictor — a CC0 Repository With a Restricted Model in It#
This is the clearest example of the chain rule in the whole survey, because both ends are published and they disagree.
The repository grant. davisking/dlib-models is CC0-1.0 and its README
opens:
“This repository contains trained models created by me (Davis King). … As far as I am concerned, anyone can do whatever they want with these model files as I’ve released them into the public domain.”
The countervailing note, in the same README, under
shape_predictor_68_face_landmarks.dat:
“This is trained on the ibug 300-W dataset … The license for this dataset excludes commercial use and Stefanos Zafeiriou, one of the creators of the dataset, asked me to include a note here saying that the trained model therefore can’t be used in a commercial product. So you should contact a lawyer or talk to Imperial College London to find out if it’s OK for you to use this model in a commercial product.”
The identical paragraph appears under
shape_predictor_68_face_landmarks_GTX.dat.
The upstream terms, at Imperial College’s own pages. The facial point annotations page:
“All the annotations are provided for research purposes ONLY (NO commercial products).”
and the 300-W challenge page:
“The data are provided for research purposes only. Commercial use (i.e., use in training commercial algorithms) is not allowed.”
Why this matters more than any other line in this file. The 68-point
landmark model is the most reproduced artifact in the category. It is what
every “facial landmarks in Python” tutorial loads, what face_recognition
uses by default, what several DeepFace paths load, and what appears in
S1’s own code samples twice. A team can carry it for years without anyone
reading the paragraph its author asked to have included.
The available alternatives, both documented in the same README:
shape_predictor_5_face_landmarks.dat, trained on King’s own dataset with
no restriction note, which is sufficient for the alignment step that most
68-point uses are really performing; or mmod_human_face_detector.dat and
the recognition ResNet, which carry no such note.
Case 2: InsightFace — MIT Code, Research-Only Weights, Stated Three Times#
InsightFace does not hide this. It states it in the repository README, in the model zoo README, and in the Python package README, and the third statement closes the loophole the first two leave.
Repository README:
“The code of InsightFace is released under the MIT License. There is no limitation for both academic and commercial usage.
>The training data containing the annotation (and the models trained with these data) are available for non-commercial research purposes only.>Both manual-downloading models from our github repo and auto-downloading models with our [python-library] follow the above license policy(which is for non-commercial research purposes only).”
Model zoo README, first line under the title:
“:bell: ALL models are available for non-commercial research purposes only.”
Python package README:
“The pretrained models we provided with this library are available for non-commercial research purposes only, including both auto-downloading models and manual-downloading models.”
The auto-download clause is the operative one. The canonical snippet —
FaceAnalysis(name='buffalo_l') — fetches a 326 MB pack on first call.
No click-through, no model card acceptance, no prompt. The weights arrive
and they are research-only.
A commercial path exists. Since a README update dated 2025-11-24:
“2. For open-sourced face recognition models (e.g., buffalo_l package), please contact [email protected] for licensing.”
with separate addresses for the inswapper face-swap models and the InspireFace SDK. So the position is not “you may not”; it is “not on these terms, ask us”. That is a negotiation, and its existence should shape how a team plans rather than whether they consider the project.
The metadata got quieter, which is a mixed result. The package’s own changelog for 1.0.1, dated 2026-05-23:
“Remove the PyPI package metadata license classifier field while keeping the README license guidance.”
So insightface on PyPI now declares no license, and GitHub reports the
repository license as null because there is no LICENSE file at the root.
An automated scanner reports unknown, which is at least not wrong — it
would have reported MIT before, which was the more misleading answer. The
substantive terms remain prose in three READMEs and are not machine-readable
anywhere. That is the specific way this trap survives procurement.
Case 3: retina-face — MIT All the Way Down, Except the Part That Matters#
The retina-face package is MIT, RetinaFace-tf2 upstream of it is MIT,
Pytorch_Retinaface is MIT. Every code repository in the chain is
permissive. And the package’s own README states where the weights came
from:
“RetinaFace is the face detection module of insightface project. The original implementation is mainly based on mxnet. Then, its tensorflow based re-implementation is published by Stanislas Bertrand. So, this repo is heavily inspired from the study of Stanislas Bertrand. Its source code is simplified and it is transformed to pip compatible but the main structure of the reference model and its pre-trained weights are same.”
Emphasis added. retinaface/model/retinaface_model.py downloads
retinaface.h5 from a release asset in serengil/deepface_models.
The chain:
InsightFace RetinaFace weights (model zoo: "non-commercial research purposes only")
→ RetinaFace-tf2 (MIT code)
→ serengil/retinaface (MIT code)
→ retinaface.h5 re-hosted on a GitHub releaseAn MIT grant on a repository covers what that repository’s authors own. Nothing in this chain is a relicensing statement about the weights, and this survey found no document from InsightFace releasing them.
This survey does not resolve the question. It records that the chain does not terminate in a permissive grant on the weights, and that S1’s licensing table — “RetinaFace: MIT (implementations), ✓ Free commercial use” — describes the code repositories and not the file that does the detecting. A team relying on that row should get the question answered rather than inherit it.
Case 4: DeepFace — Where the License Is a Configuration Value#
DeepFace’s own README is the most useful document in this category on this subject:
“DeepFace wraps some external face recognition models: VGG-Face, Facenet (both 128d and 512d), OpenFace, DeepFace, DeepID, ArcFace, Dlib, SFace, GhostFaceNet and Buffalo_L. … Similarly, DeepFace wraps many face detectors: OpenCv, Ssd, Dlib, MtCnn, Fast MtCnn, RetinaFace, MediaPipe, YuNet, Yolo and CenterFace. … License types will be inherited when you intend to utilize those models. Please check the license types of those models for production purposes.”
Emphasis in the original.
The consequence is that these two calls have different license positions and identical shapes:
| Configuration | Resulting position |
|---|---|
detector_backend="yunet", model_name="SFace" | MIT detector, Apache-2.0 recognizer — clear |
detector_backend="opencv", model_name="Dlib" | Intel-licensed cascade; dlib’s model chain applies |
detector_backend="yolov8" | AGPL-3.0, via ultralytics/ultralytics |
model_name="Buffalo_L" | Research only, via InsightFace |
Two string arguments determine whether your service is permissively licensed, network-copyleft, or non-commercial. No dependency scanner will tell you which, because all four configurations have the identical dependency tree.
The practical control: pin detector_backend and model_name explicitly
in code, never rely on the defaults, and treat those two strings as
license-bearing configuration reviewed like a dependency.
Case 5: The Attribution Notices Nobody Generates#
Two smaller findings for teams that generate their third-party notices file mechanically from repository licenses:
- The Haar cascade XML is not under OpenCV’s Apache-2.0. Its header is
an “Intel License Agreement / For Open Source Computer Vision Library /
Copyright (C) 2000, Intel Corporation, all rights reserved” — a
BSD-shaped grant with a no-endorsement clause naming Intel. Permissive,
redistributable, and a different notice from the one in
LICENSE. ShiqiYu/libfacedetection, upstream of YuNet, resolves toNOASSERTIONon GitHub because its LICENSE is a custom-formatted 3-clause BSD text rather than a recognized template. The redistributed ONNX inopencv_zoocarries its own MITLICENSE; the upstream is BSD-3. A scanner reporting “unknown license” for libfacedetection is reporting a parsing result, not a licensing problem.
The Practical Checklist#
For any face model entering a build:
- Find the weights file. Not the package — the
.dat,.onnx,.h5or.taskthat the code loads. Look in~/.insightface/models/,~/.deepface/weights/, and whereverdlib.shape_predictorpoints. - Find the license for that file. A
LICENSEbeside it in the repository is the good case (YuNet, SFace). A README paragraph is the common case (InsightFace, dlib). Nothing at all is the case that needs the most work (retinaface.h5). - Read what the model was trained on, and check the dataset’s own terms at the institution that publishes it. dlib’s README is the model for how a project should document this.
- Check whether the package auto-downloads. If it does, the terms attached to the download are terms you accepted by running the code.
- Pin the file, not the package. Version and hash. Then the license answer stays true across rebuilds.
Sources#
- davisking/dlib-models README — CC0 grant and the 300-W note
- iBUG facial point annotations; iBUG 300-W
dlib/image_processing/frontal_face_detector.h— Boost header, serialized weights- deepinsight/insightface README; model zoo; python-package README and changelog
- serengil/retinaface README and
retinaface/model/retinaface_model.py - serengil/deepface README
- opencv_zoo YuNet LICENSE (MIT); SFace LICENSE (Apache-2.0)
opencv/opencvdata/haarcascades/haarcascade_frontalface_default.xml— Intel License Agreement header- ShiqiYu/libfacedetection LICENSE — 3-clause BSD
- ultralytics/ultralytics — AGPL-3.0
Maintenance and Durability: Which of These Will Still Be Here#
Measured 2026-08-25 via the GitHub REST API and the PyPI JSON API. Every figure below is reproducible with two GitHub search queries and one PyPI request per project.
The Whole Table#
| Project | Stars | Last release | Last commit | Open PRs | Merged since 2026-02-25 | Open issues | 30-day downloads |
|---|---|---|---|---|---|---|---|
opencv/opencv | 90,601 | 5.0.0.93 / 4.14.0.94, 2026-07 | 2026-08-25 | 208 | 600 | 2,758 | 53,780,796 |
google-ai-edge/mediapipe | 36,723 | 1.0.1, 2026-08-14 | 2026-08-25 | 199 | 0 † | 551 | 3,022,607 |
deepinsight/insightface | 29,569 | 1.0.1, 2026-05-23 | 2026-07-27 | 43 | 3 | 1,266 | 1,081,113 |
serengil/deepface | 23,337 | 0.0.100, 2026-05-09 | 2026-08-24 | 2 | 11 | 9 | 487,770 |
davisking/dlib | 14,431 | 20.0.1, 2026-03-29 | 2026-08-11 | 2 | 11 | 38 | 198,251 |
serengil/retinaface | 2,027 | 0.0.18, 2026-06-01 | 2026-06-01 | 0 | 1 | 1 | 178,178 |
ipazc/mtcnn | 2,484 | 1.0.0, 2024-10-08 | 2024-10-08 | 0 | 0 | 28 | 256,107 |
ageitgey/face_recognition | 56,682 | 1.3.0, 2020-02-20 | 2026-06-25 ‡ | 57 | 1 | 832 | 186,937 |
timesler/facenet-pytorch | 5,162 | 2.6.0, 2024-04-29 | 2024-08-02 | 7 | 0 | 85 | 135,604 |
davisking/dlib-models | 1,617 | — | 2025-05-28 | — | — | 22 | — |
opencv/opencv_zoo | 1,033 | — | 2026-05-28 | — | — | 31 | — |
biubug6/Pytorch_Retinaface | 2,976 | — | 2020-04-20 | — | — | 148 | — |
† See “The False Positive” below. ‡ The commit is a merge of a contributed link fix; see “The Most-Starred Dead Project”.
What the Stall Test Found#
ADDING-RESEARCH’s stall test compares open pull requests against merges over a window: patches arriving with nobody merging them fires months before an archive notice. Applied here it produced three distinct results, and two of them are shapes the test does not name.
The healthy shape#
OpenCV: 208 open, 600 merged in six months, a commit on the day of the measurement. Near 3:1 in favor of merges. This is what a live project looks like and it is the largest, most-installed thing in the survey.
DeepFace and dlib are the small-project version of the same shape: 2 open against 11 merged each, empty-ish issue trackers, recent commits. DeepFace’s 9 open issues against 23,337 stars is the tidiest tracker here by a wide margin.
serengil/retinaface is quieter still — 0 open, 1 merged, 1 open issue
— which for a package that does one thing and does it is completeness
rather than neglect.
The false positive#
MediaPipe returns 199 open pull requests and 0 merged in six months. Read against the test alone, that is the failure signature.
It is not a failure. The last commit on master was made on the day of
this measurement, and the v1.0.0 release notes list dozens of substantive
internal changes — Abseil updated from 2023-10-18 to 2025-01-14, WebGPU
service work, LiteRT accelerator documentation, EXIF orientation support in
the Python Task API, a null-pointer fix in Tensor::GetOpenGlTexture2dReadView().
MediaPipe is developed inside Google and synced out. Community pull
requests accumulate because they are not the input path.
The generalizable correction: for a project developed internally and mirrored to GitHub, compare open PRs against commits, not against merges. Zero merges plus daily commits is a governance shape. Zero merges plus zero commits is a stall.
What the number does tell an adopter is real, and it is not nothing: an outside patch is unlikely to land. A MediaPipe bug you need fixed is a bug you wait for or work around, and 199 people have already discovered that.
The shape the test does not have a name for: finished#
ipazc/mtcnn: 0 open pull requests, 0 merged, last commit 2024-10-08,
28 open issues, 256,107 downloads a month.
Zero merges is half the stall signal. Zero open pull requests is the other half missing — nothing is arriving either. The stall test fires on a ratio, on patches queueing up unmerged. There is no queue here.
This is a finished project: a 2016 algorithm, packaged, released as 1.0.0 in October 2024, and left alone. The maintainer’s release history — 0.1.0 in 2019, 0.1.1 in 2021, 1.0.0 in 2024 — is someone who returns for major work occasionally, not someone who walked away mid-sentence.
The risk is different from abandonment risk. Nothing will break on its own; nothing will be fixed either. Treat it as a frozen dependency with a known expiry set by whenever TensorFlow next breaks it.
Proposed addition to the stall vocabulary, for this library’s use:
| Open PRs | Merges | Commits | Reading |
|---|---|---|---|
| Many | Many | Many | Healthy |
| Many | Zero | Many | Mirrored — internal development, outside patches do not land |
| Many | Zero | Zero | Stalled — the signal the test was built for |
| Zero | Zero | Zero | Finished — or forgotten; check the release history to tell them apart |
The Most-Starred Dead Project in the Category#
ageitgey/face_recognition deserves its own section because it is the
single most likely thing a reader already depends on and the most likely
thing to pass a shallow health check.
- 56,682 stars — more than InsightFace, more than MediaPipe, more than dlib, more than anything else in this survey.
- 13,699 forks.
- Last PyPI release: 1.3.0, 2020-02-20. Six and a half years.
- 57 open pull requests, 1 merged since 2026-02-25.
- 832 open issues.
- Last commit: 2026-06-25 — and that commit is
Merge pull request #1672 from 17X61/auto-contrib/1670-lfw-link, a repair to a broken link.
pushed_at is two months old. A freshness check that reads pushed_at
records this project as active. The substance is one merged link fix
against 57 waiting patches.
The three signals that matter here disagree with each other, which is the lesson:
| Signal | What it says | Correct reading |
|---|---|---|
| Stars (56,682) | Very healthy | Stars are cumulative and never decay |
pushed_at (2026-06-25) | Recent activity | One contributed link fix |
| Last release (2020-02-20) | Stopped | Correct |
| 57 open / 1 merged | Stopped | Correct |
Stars are the least informative signal in this category and the first one
a reader sees. The two dormant projects here — face_recognition and
facenet-pytorch — hold 61,844 stars between them against the maintained
DeepFace’s 23,337.
The other thing this project demonstrates is inherited fragility.
face_recognition depends on dlib, which has no wheels on PyPI; the
combination is a package nobody is fixing on top of a build nobody can
avoid. Every “face_recognition won’t install” thread is a CMake thread, and
57 of the fixes are written and waiting.
Concentration and Governance Risk#
| Project | Backing | Concentration risk |
|---|---|---|
| OpenCV | OpenCV Foundation, corporate contributors, 56,972 forks | Lowest in the survey |
| MediaPipe | Google, developed internally | Product risk, not abandonment risk — Google retires products |
| InsightFace | A research group moving commercial | The models are the funnel; terms have already tightened once |
| dlib | One person, Davis King | Highest bus factor; dlib-models untouched since 2025-05-28 |
| DeepFace / retina-face | One person, Sefik Ilkin Serengil | High bus factor, currently well managed |
| MTCNN | One person, finished | Already realized |
| face_recognition | One person, stopped | Already realized |
The two single-maintainer projects that are healthy today are healthy because of one person each, and both of them are doing a good job. That is not a criticism; it is the risk, and it is a different risk from the one the stall test measures. dlib is thirteen years old and shows no sign of stopping, and it would stop the day Davis King stopped.
InsightFace’s risk is directional rather than existential. The project added per-model licensing contacts on 2025-11-24, shipped a commercial evaluation GUI in May 2026 whose documentation names “a commercial model license” as the destination, and announced a hosted-server product in July 2026. Terms that have tightened once can tighten again. A team depending on the current research-only-but-free position should note that the current position is a choice the project is actively revisiting.
MediaPipe’s risk is that Google retires things. The project’s own
history shows the shape: the legacy Solutions API still ships alongside the
Tasks API that replaced it, and the repository moved organizations from
google/ to google-ai-edge/. Neither event broke anything. A third such
transition might.
Version Cadence, and What Moved Since S1#
S1 was written in January 2025. Everything below happened after it:
- InsightFace 1.0 and 1.0.1, both 2026-05-23, after three years at 0.7.3. First wheel; PyPI license classifier removed; enterprise evaluation GUI added.
- InsightFace licensing contacts added to the README 2025-11-24.
- InsightFace Server announced 2026-07-27 as “a simple AWS Rekognition alternative”.
- MediaPipe 1.0.0, 2026-07-27, after years on 0.10.x. 1.0.1 followed 2026-08-14.
- OpenCV 5.0 shipped (5.0.0.93 on PyPI, 2026-07-02) and the repository
default branch moved to
5.x. The 4.x line continues — 4.14.0.94 was uploaded later, on 2026-07-29. - dlib 20.0 (2025-05-28) and 20.0.1 (2026-03-29).
facenet-pytorchwent quiet: last commit 2024-08-02.opencv_zooaddedface_detection_yunet_2026may.onnxwith dynamic input dims for the OpenCV 5 ONNX Runtime engine.
Three of the four largest projects in this category shipped a major version
in the eighteen months after S1 was written. This is a fast-decay category
and the decay_class: fast in its metadata is right.
Durability Verdict#
| Project | Verdict |
|---|---|
| OpenCV | Safest bet in the category. Foundation-backed, 600 merges in six months, two supported lines |
| MediaPipe | Durable while Google wants it. Outside patches do not land |
| InsightFace | Alive and moving commercial. Terms have tightened once |
| dlib | Stable, mature, one maintainer, thirteen years of evidence |
| DeepFace | Well maintained, one maintainer, tidiest tracker here |
| retina-face | Small and complete; same maintainer as DeepFace |
| MTCNN | Finished. Frozen, not broken |
face_recognition | Stopped. Six years without a release, 57 patches waiting |
facenet-pytorch | Dormant. Two years without a commit, pinned against a moving PyTorch |
Pytorch_Retinaface | Dead. Last commit 2020-04-20, 148 issues open |
Sources#
All figures from the GitHub REST API (/repos/{owner}/{name},
/repos/{owner}/{name}/commits) and the GitHub search API
(type:pr state:open, type:pr is:merged merged:>2026-02-25), the PyPI
JSON API, and pypistats.org, all queried 2026-08-25. The stall-test method
is from ADDING-RESEARCH.md.
S4 Verdict: Strategic Paths#
Date: 2026-08-25.
The Organizing Finding#
In this category, what you are allowed to ship reorders the leaderboard, and the reordering is not close.
The highest-scoring recognition model is licensed for non-commercial research only, by its project’s own statement, repeated in three separate documents. The most reproduced landmark model in the world carries a note from its author saying it cannot be used in a commercial product. The pip package that redistributes the best-known detector’s weights is MIT over an upstream that is not, and nothing in the chain says otherwise.
Meanwhile the one detection-plus-recognition pair whose weights carry
license files permitting commercial use — YuNet and SFace, MIT and
Apache-2.0, 38 MB together — is already inside opencv-python, and S1
mentions one of them twice in passing and the other not at all.
The second finding follows from the first. Detection, landmarks and
recognition are three jobs, and the third one is the only one that is
biometric identification in the sense every regime in regulatory-surface.md
cares about. A feature that can be built with detection alone avoids the
license problem, the demographic-accuracy problem, the re-enrollment
problem and most of the regulatory surface at once. Engineers reach for
recognition by default because it is the impressive part.
Four Strategic Paths#
Path 1 — The permissive stack#
cv2.FaceDetectorYN + cv2.FaceRecognizerSF.
For anyone shipping a commercial product who cannot negotiate model
licenses. Both models have a LICENSE file beside them in
opencv/opencv_zoo — MIT and Apache-2.0. Both are callable from the
OpenCV the project already imports. YuNet is 227 KB, or 98 KB quantized
with no measurable loss on the hard subset. SFace is 37 MB. They are
designed as a pair: YuNet emits the five landmarks SFace’s alignment wants.
The underlying project is the healthiest in the survey — 600 pull requests merged in six months, Apache-2.0, a foundation behind it.
What you accept: SFace publishes one number on one saturated benchmark, and nothing on IJB-C, MR-ALL or demographic subsets. The accuracy gap to InsightFace exists and is unmeasured publicly. Say so when asked, rather than implying parity.
Path 2 — The geometry stack#
MediaPipe Face Landmarker.
For on-device face geometry: AR, avatars, expression, gaze, virtual try-on. 478 3D landmarks, 52 blendshape coefficients and a facial transformation matrix from one bundle, with first-party Android, iOS and Web bindings. No other option in the category does this job, so the comparison is not against alternatives but against building it yourself.
The license chain came back clean end to end, which for a consumer app release is the whole game.
What you accept: no recognition, ever — MediaPipe has no embedding model. Outside patches do not land (199 open pull requests, 0 merged in six months). No macOS x86_64 wheel at 1.0.1.
Path 3 — The research stack#
InsightFace, with the encoder behind an interface.
For research, benchmarking and internal study, where “non-commercial research purposes only” describes the use exactly. It is the only project publishing results on the benchmarks that separate models — IJB-C, CFP-FP, AgeDB-30, MR-ALL and the demographic splits — and the only one publishing demographic breakdowns at all.
The discipline that makes this path safe is one rule: keep the encoder
behind encode() and compare() from the first commit. Then the day the
work becomes commercial, the licensing conversation with
[email protected] is a decision with a fallback, rather
than a rewrite with a re-enrollment attached.
What you accept: 1,266 open issues and 3 merged pull requests in six
months. No LICENSE file at the repository root, so scanners report
unknown. A project moving commercial whose terms have already tightened
once.
Path 4 — The hosted path#
Amazon Rekognition, at $1.00 per 1,000 images in the first tier, for teams with no ML capacity and moderate volume.
What you accept: face images leave your infrastructure, which is a data-protection decision before it is an engineering one; the enrolled gallery lives in the vendor’s Collection and cannot be exported; and the migration back out is a re-enrollment, not a config change. S1 presents “cloud for the MVP, self-host at scale” as cost optimization with a break-even point. The break-even arithmetic is the easy part.
Note what the market says about this path: Microsoft sells detection to anyone and will not sell identification without an account team and a stated use case, and has prohibited use by or for US police departments since 2020-06-11. That is one of three major clouds declining revenue in this category.
The Rules That Survive Any Choice#
1. Audit the weights, not the package. Find the .dat, .onnx, .h5
or .task the code loads. Find its license — a file beside it in the
repository if you are lucky, a README paragraph if not, nothing at all in
one case here. Read what it was trained on and check the dataset’s own
terms at the institution that publishes it. A dependency scanner reads the
first link of a three-link chain.
2. The encoder is the irreversible decision. Detectors swap in an afternoon. Landmark conventions cost a rewrite of the consumer. Encoders, once templates are stored, cost every enrolled person re-enrolling. Interface the encoder; store the model name, version and hash alongside every template.
3. Detection-only is a real answer and is underused. It avoids the license problem, the demographic problem, the re-enrollment problem and most of the regulatory surface. Ask whether the feature needs identity before assuming it does.
4. Three questions come before the model. Do we store a template? Does it leave our infrastructure? Is this verification or identification? Every regime this survey read turns on some combination of those, and all three are settled by engineering decisions made before a benchmark is consulted.
5. Measure on your own population. No open-source model in this survey participates in NIST FRTE, so every figure here is self-reported. The number that decides your deployment — false match and false non-match rate at your operating point, on your users, with your gallery size — is not on any of these tables and never will be.
6. Two string arguments can change your license. If DeepFace is in the
build, detector_backend and model_name span MIT, Apache-2.0, AGPL-3.0
and research-only, with an identical dependency tree in every case. Pin
them explicitly and review them like dependencies.
What Would Change This Survey#
Recorded so the next refresh knows what to watch:
- InsightFace relicensing
buffalo_l. The project added a licensing contact for it in November 2025 and shipped commercial evaluation tooling in May 2026. If open commercial terms ever arrive, Path 1 and Path 3 collapse into one and the category changes shape. - SFace or YuNet getting harder benchmark numbers. The strongest argument against Path 1 today is an absence of data, not a measured deficit. An IJB-C or MR-ALL evaluation of SFace would settle it.
- A permissively-licensed landmark model at 68 points or better. The 300-W restriction has shaped this category for a decade. A replacement trained on unencumbered data would retire the most common trap in it.
- The EU AI Act’s amended timeline. An amending package moving Annex III high-risk obligations later has been widely reported; this survey could not verify its Official Journal publication and does not rely on it.
dlibshipping wheels. It would remove the largest practical friction with the most mature stack in the category.- Any of these models entering NIST FRTE. It would give the category its first independent numbers.
What This Survey Does Not Answer#
- Whether the RetinaFace weights redistributed through
retina-facecarry InsightFace’s non-commercial restriction. The code chain is MIT end to end; no relicensing statement for the weights was found anywhere in it. Unresolved is not the same as permitted. - Face++ per-call pricing. The table renders client-side and was not machine-readable.
- Comparative latency on identical hardware. No verifiable cross-project benchmark exists. Producing one is a build.
- Anything about how any of these rules will be enforced. The regulatory material reports text and says where it was read. It is not legal advice, and a deployment decision needs counsel.
The Regulatory Surface: Where This Question Arises#
This file reports what three regimes say and where each text was read. It is not legal advice. It does not predict how any rule will be applied to any deployment, and it makes no claim about enforcement. A real decision needs counsel who has seen the specifics.
The useful framing is the one this library uses for the AGPL: where does this question arise? Which architectural choices put a system inside a rule’s scope, and which keep it outside. That is answerable from the text of the rules, and it is the part that belongs in a software survey, because it changes which library you choose and how you use it.
Everything was read on 2026-08-25.
The Distinction Every Regime Draws#
Before any statute: the three jobs in this category are not equally regulated, and the boundary that matters most is between them.
| Operation | What it produces | Regulatory weight |
|---|---|---|
| Detection | A bounding box: a face is present, here | Lowest. Nothing identifies anyone |
| Landmarks | Geometry: eye corners, mesh, expression | Low for geometry; emotion and sensitive-attribute inference are named directly in EU law |
| Verification (1:1) | Is this the person they claim to be? | Middle. Carved out of some rules |
| Identification (1:N) | Who is this, out of a gallery? | Highest, everywhere |
The market draws this line as well as the law does. Microsoft sells face detection to anyone and gates identification and verification behind a vetting process. Texas’s 2026 amendment turns an AI-training exemption on whether a system “is used or deployed for the purpose of uniquely identifying a specific individual”. Annex III of the EU AI Act carves verification out of remote biometric identification.
Practical consequence: a feature that can be delivered with detection alone avoids most of what follows. Engineers reach for recognition by default because it is the impressive part; a large share of proposals do not need it.
Illinois: the Biometric Information Privacy Act (740 ILCS 14)#
Read on 2026-08-25 from the Illinois General Assembly’s own publication of the Compiled Statutes.
What it covers#
Section 10:
“‘Biometric identifier’ means a retina or iris scan, fingerprint, voiceprint, or scan of hand or face geometry. Biometric identifiers do not include writing samples, written signatures, photographs, human biological samples used for valid scientific testing or screening, demographic data, tattoo descriptions, or physical descriptions such as height, weight, hair color, or eye color.”
Emphasis added. Photographs are excluded from the definition of a biometric identifier; a “scan of … face geometry” is not. The same section then defines biometric information broadly:
“‘Biometric information’ means any information, regardless of how it is captured, converted, stored, or shared, based on an individual’s biometric identifier used to identify an individual.”
“Regardless of how it is captured, converted, stored, or shared” is the clause that reaches embeddings. An embedding is not a photograph and it is derived from face geometry and used to identify.
The Act applies to a “private entity”, defined to exclude “a State or local government agency” and Illinois courts. Section 25(c) excludes financial institutions subject to Title V of the Gramm-Leach-Bliley Act, and 25(e) excludes contractors and agents of a State agency or local government when working for them.
What it requires#
Section 15(a) — a published retention policy:
“A private entity in possession of biometric identifiers or biometric information must develop a written policy, made available to the public, establishing a retention schedule and guidelines for permanently destroying biometric identifiers and biometric information when the initial purpose for collecting or obtaining such identifiers or information has been satisfied or within 3 years of the individual’s last interaction with the private entity, whichever occurs first.”
Section 15(b) — notice and a written release before collection:
“No private entity may collect, capture, purchase, receive through trade, or otherwise obtain a person’s or a customer’s biometric identifier or biometric information, unless it first: (1) informs the subject … in writing that a biometric identifier or biometric information is being collected or stored; (2) informs the subject … in writing of the specific purpose and length of term for which a biometric identifier or biometric information is being collected, stored, and used; and (3) receives a written release executed by the subject…”
Section 15(c) forbids selling, leasing, trading or otherwise profiting from it. Section 15(d) forbids disclosure except on four grounds — consent, completing a requested financial transaction, a legal requirement, or a warrant. Section 15(e) requires storage using “the reasonable standard of care within the private entity’s industry” and at least as protective as how the entity treats its other confidential and sensitive information.
Why it is the one people mean#
Section 20 gives a private right of action:
“Any person aggrieved by a violation of this Act shall have a right of action … A prevailing party may recover for each violation: (1) against a private entity that negligently violates a provision of this Act, liquidated damages of $1,000 or actual damages, whichever is greater; (2) against a private entity that intentionally or recklessly violates a provision of this Act, liquidated damages of $5,000 or actual damages, whichever is greater; (3) reasonable attorneys’ fees and costs…”
That is why BIPA is the statute engineers have heard of. Individuals can sue; damages are liquidated; fees are recoverable.
The 2024 amendment#
Public Act 103-769, effective 2024-08-02, added subsections (b) and (c) to Section 20:
“…a private entity that, in more than one instance, collects, captures, purchases, receives through trade, or otherwise obtains the same biometric identifier or biometric information from the same person using the same method of collection in violation of subsection (b) of Section 15 has committed a single violation … for which the aggrieved person is entitled to, at most, one recovery…”
with a parallel rule for repeated disclosure to the same recipient. The same act added “electronic signature” to the definition of “written release”.
The engineering reading: the per-scan multiplication is capped per person per collection method. The notice-and-release requirement is unchanged, and “electronic signature” being named makes an in-product consent flow a recognized mechanism.
Where the question arises#
- Storing a face template of an Illinois resident, for any purpose
- Collecting one without a prior written notice and release
- Having no published retention and destruction policy
- Retaining a template past purpose satisfaction, or past 3 years from last interaction
- Disclosing templates to a vendor without consent — which includes sending images to a hosted API, depending on how the arrangement is structured
Texas: Capture or Use of Biometric Identifier (Tex. Bus. & Com. Code § 503.001)#
Read on 2026-08-25. Sourcing note: statutes.capitol.texas.gov now serves a JavaScript application that returns no statutory text to a non-browser client, so this text was read from an archived capture of that official page dated 2025-10-04, which carries the amendment notices described below. This is weaker sourcing than the Illinois text and a refresh should re-read the live page in a browser.
What it covers#
Subsection (a) uses almost the same definition as Illinois — “a retina or iris scan, fingerprint, voiceprint, or record of hand or face geometry” — and the section applies to capture “for a commercial purpose”.
What it requires#
Subsection (b): no capture for a commercial purpose without informing the individual beforehand and receiving consent. Subsection (c): no sale, lease or disclosure except on four enumerated grounds; storage with reasonable care at least as protective as the possessor’s other confidential information; and destruction
“within a reasonable time, but not later than the first anniversary of the date the purpose for collecting the identifier expires”
Subsection (c-2) settles a common employment case:
“If a biometric identifier captured for a commercial purpose has been collected for security purposes by an employer, the purpose for collecting the identifier under Subsection (c)(3) is presumed to expire on termination of the employment relationship.”
Who enforces it#
Subsection (d):
“A person who violates this section is subject to a civil penalty of not more than $25,000 for each violation. The attorney general may bring an action to recover the civil penalty.”
No private right of action. That is the structural difference from Illinois, and it changes the shape of the risk rather than its existence.
The 2026 amendment: scraping and AI training#
Acts 2025, 89th Leg., R.S., Ch. 1174 (H.B. 149), effective 2026-01-01, made three changes that bear directly on this category.
Scraped images are not consent. New subsection (b-1):
“For purposes of Subsection (b), an individual has not been informed of and has not provided consent for the capture or storage of a biometric identifier of an individual for a commercial purpose based solely on the existence of an image or other media containing one or more biometric identifiers of the individual on the Internet or other publicly available source unless the image or other media was made publicly available by the individual to whom the biometric identifiers relate.”
An AI-training exemption, with the identification carve-out. New subsection (e)(2) exempts
“the training, processing, or storage of biometric identifiers involved in developing, training, evaluating, disseminating, or otherwise offering artificial intelligence models or systems, unless a system is used or deployed for the purpose of uniquely identifying a specific individual”
with (e)(3) exempting development or deployment for security-incident, fraud, identity-theft and related purposes.
And a snap-back. New subsection (f): if an identifier captured for AI training “is subsequently used for a commercial purpose not described by Subsection (e)”, the section’s possession and destruction provisions and penalties apply.
Subsection (a) also gained a definition of “artificial intelligence system” by cross-reference to § 551.001.
Where the question arises#
- Capturing face geometry for a commercial purpose in Texas without prior notice and consent
- Assuming that images scraped from the public internet carry consent — the 2026 amendment says they do not, unless the subject published them
- Building a training set from web images and then deploying the resulting system to identify specific individuals, which puts you back inside the section by (e)(2)’s own terms
- Retaining an employee’s face template after they leave, which (c-2) addresses directly
Relevance to this survey’s own material: dlib’s recognition model was trained, per its author’s README, on “the face scrub dataset …, the VGG dataset …, and then a large number of images I scraped from the internet.” Several other models in the category were trained on web-scale face datasets. That is a fact about the models a reader is choosing among.
The European Union#
GDPR Article 9#
“Processing of personal data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and the processing of genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person’s sex life or sexual orientation shall be prohibited.”
Emphasis added. Two things follow that the engineering discussion routinely misses.
“For the purpose of uniquely identifying” is the trigger, not the data type. Detecting a face is processing personal data; computing a template to identify someone is processing special-category data. The same camera frame can be either, depending on what you do next.
On-premises processing does not exit Article 9. S1’s characterization of self-hosted libraries as “GDPR-compliant” and “Privacy-Friendly” is wrong in a way that matters, because it is the sentence engineers quote to compliance owners. Running the model on your own hardware removes the international transfer question and the processor relationship. The prohibition in Article 9(1) and the need for an Article 9(2) condition are unchanged.
The AI Act — Regulation (EU) 2024/1689#
Sourcing note: the article text below was read on 2026-08-25 from the AI Act Explorer reproduction at artificialintelligenceact.eu. EUR-Lex returned an HTTP 202 challenge page to every automated request made for this survey, so the Official Journal original was not machine-read. This is weaker sourcing than the rest of this pass and a refresh should re-read these articles from the OJ.
Article 5(1) — prohibited practices. Three concern biometrics, and two of them catch ordinary product features:
(f) prohibits
“the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons”
(g) prohibits
“the placing on the market, the putting into service for this specific purpose, or the use of biometric categorisation systems that categorise individually natural persons based on their biometric data to deduce or infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation”
(h) prohibits
“the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purposes of law enforcement, unless and in so far as such use is strictly necessary for one of the following objectives:
>(i) the targeted search for specific victims of abduction, trafficking in human beings or sexual exploitation of human beings, as well as the search for missing persons;>(ii) the prevention of a specific, substantial and imminent threat to the life or physical safety of natural persons or a genuine and present or genuine and foreseeable threat of a terrorist attack;>(iii) the localisation or identification of a person suspected of having committed a criminal offence…”
Two of these are checkboxes in libraries in this survey. DeepFace ships emotion inference and a race/ethnicity attribute model; the hosted APIs ship emotion inference. A team that enabled those because they were available should reread (f) and (g).
Annex III point 1 — high-risk classification. Biometrics are listed as high-risk “in so far as their use is permitted under relevant Union or national law”, covering: remote biometric identification systems, with an exclusion for systems intended for “biometric verification the sole purpose of which is to confirm that a specific natural person is the person he or she claims to be”; biometric categorisation according to sensitive or protected attributes; and emotion recognition.
The verification exclusion is the practical one for product teams. Unlocking a device by matching a face against a single template on that device is verification. Scanning a space against a gallery is remote identification. Same encoder; different classification.
Article 113 — application dates:
“It shall apply from 2 August 2026.”
with
“(a) Chapters I and II shall apply from 2 February 2025; (b) Chapter III Section 4, Chapter V, Chapter VII and Chapter XII and Article 78 shall apply from 2 August 2025, with the exception of Article 101; (c) Article 6(1) and the corresponding obligations in this Regulation shall apply from 2 August 2027.”
The Article 5 prohibitions sit in Chapter II and have applied since 2 February 2025.
Not verified. An amending “Digital Omnibus” package moving the Annex III high-risk obligations later has been widely reported by law firms and trade press. This survey could not confirm its publication in the Official Journal and does not rely on it. A reader for whom the date is decisive should check the OJ directly.
Vendor Behavior as a Signal#
Two facts about how the market has responded, both verified, both useful to a reader weighing self-hosted against hosted.
Microsoft gates identification. From Microsoft’s Limited Access documentation:
“Customers and partners who wish to use Limited Access features of the Face API, including Face identification and Face verification, are required to register for access by submitting a registration form. … The Face Detection operation is available without registration.”
and
“Face API is available only to customers managed by Microsoft, defined as those customers and partners who are working directly with Microsoft account teams.”
and
“Since the announcement on June 11th, 2020, Azure Face recognition in Foundry Tools is strictly prohibited for use by or for U.S. police departments.”
One of the three major clouds will not sell face identification to a developer with a credit card. That is a market fact with more information in it than any accuracy table.
Megvii, the company behind Face++, is on the US Entity List. The Bureau of Industry and Security’s final rule published 2019-10-09 (Federal Register document 2019-22210) states:
“The following eight entities are also added to the Entity List as part of this rule: Dahua Technology; Hikvision; IFLYTEK; Megvii Technology; Sense Time, Xiamen Meiya Pico Information Co. Ltd.; Yitu Technologies; and Yixin Science and Technology Co. Ltd.”
The rule’s stated basis for the twenty-eight entities it adds is that they “have been implicated in human rights violations and abuses in the implementation of China’s campaign of repression, mass arbitrary detention, and high-technology surveillance against Uighurs, Kazakhs, and other members of Muslim minority groups” in Xinjiang, and it imposes “a license requirement for all items subject to the EAR” for transactions involving them.
Reported because many procurement processes screen against the list and a Face++ dependency will surface there. What the listing means for a particular commercial relationship is a question for counsel; this survey does not answer it.
The Wider Picture, Briefly#
Illinois, Texas and Washington were the early dedicated biometric statutes. Since then most US comprehensive state privacy laws have brought biometric data in as “sensitive data” requiring opt-in consent, and several jurisdictions have added sector-specific rules. This survey verified three regimes and does not attempt a fifty-state survey; a reader operating nationally should assume the surface is broader than what is documented here and get it mapped properly.
The stable part — the part that will still be true after the next amendment — is the architectural question:
- Do we store a template?
- Does it leave our infrastructure?
- Is this verification or identification?
Every regime read for this survey turns on some combination of those three, and all three are settled by engineering decisions made before a model is chosen.
Sources#
- 740 ILCS 14, Biometric Information Privacy Act — Illinois General Assembly, read 2026-08-25; amendment history P.A. 95-994 (eff. 10-3-08) and P.A. 103-769 (eff. 8-2-24) as shown in the source notes
- Tex. Bus. & Com. Code § 503.001 — read 2026-08-25 from an archived capture of statutes.capitol.texas.gov dated 2025-10-04; source note: Added by Acts 2007, 80th Leg., R.S., Ch. 885 (H.B. 2278); amended by Acts 2009, 81st Leg., Ch. 1163 (H.B. 3186); Acts 2017, 85th Leg., Ch. 913 (S.B. 1343); Acts 2025, 89th Leg., R.S., Ch. 1174 (H.B. 149), eff. January 1, 2026
- Regulation (EU) 2024/1689, Articles 5, 6, 113 and Annex III — read 2026-08-25 via artificialintelligenceact.eu; EUR-Lex returned HTTP 202 to automated requests
- GDPR Article 9
- Limited Access features of Face — Microsoft Learn (page metadata: ms.date 2025-10-15, updated 2026-06-05)
- Federal Register 2019-22210, “Addition of Certain Entities to the Entity List” — full text read from the Federal Register’s own text endpoint