Facial Recognition
How it works
The facial recognition pipeline has four main stages. (1) Face detection โ locating faces in the image (e.g. the Viola-Jones cascade or MTCNN/RetinaFace networks). (2) Alignment โ geometric normalization of the face relative to detected landmarks (eyes, nose, mouth) to reduce pose variance. (3) Embedding extraction โ a deep convolutional network maps the aligned face to a fixed-length vector (e.g. 128 or 512 dimensions), trained with a metric loss (triplet loss in FaceNet) or a margin-based softmax loss (ArcFace, CosFace, SphereFace) so that embeddings of the same person are close and different people are far apart. (4) Comparison โ for 1:1 verification a distance (cosine or Euclidean) is computed and compared to a threshold; for 1:N identification the query embedding is compared against a gallery of known identities and the nearest is selected (or rejected if it exceeds the threshold).
Problem solved
Automatic, contactless establishment or confirmation of a person's identity from a face image under varying illumination, pose, expression, and aging. Classical hand-crafted-feature methods generalized poorly; learned embeddings solve the open-set identity problem without retraining the model for each new person.
Components
Locates faces in the image, returning bounding boxes and often facial landmarks. Examples: Viola-Jones, MTCNN, RetinaFace.
Official
Normalizes face geometry relative to landmarks (affine warp to a canonical layout), reducing pose variance.
Official
A deep convolutional network (e.g. ResNet, Inception) mapping the aligned face to a fixed-length feature vector. Trained with a metric or margin-based loss.
Official
Computes similarity (cosine or Euclidean distance) between embeddings and decides: 1:1 verification against a threshold or 1:N identification via nearest-neighbor search in the gallery.
Official
Implementation
Models exhibit uneven accuracy across demographic groups (gender, ethnicity, age), as confirmed e.g. by NIST FRVT studies.
Systems without liveness detection can be fooled by a photo, video, or 3D mask (presentation attack).
A poorly chosen threshold yields an unfavorable trade-off between false acceptance (FAR) and false rejection (FRR).
Landmark detection errors or large pose deviations degrade embedding quality.
Evolution
Turk and Pentland introduce Eigenfaces โ the first practical automated face recognition method based on PCA.
The Viola-Jones cascade enables real-time face detection, becoming a foundation for practical systems.
Facebook presents DeepFace โ a deep CNN reaching 97.35% on LFW, close to human level.
Google introduces FaceNet with triplet loss and 128-dimensional embeddings, reaching 99.63% on LFW.
Zhang et al. propose MTCNN โ a CNN cascade jointly performing face detection and landmark alignment.
Deng et al. introduce Additive Angular Margin Loss (ArcFace), significantly increasing embedding discriminability.
Hyperparameters (configurable axes)
Length of the face feature vector. Trade-off between discriminability and memory/comparison cost.
Margin added in the loss function (ArcFace/CosFace) to increase separation between identity classes.
Scaling factor applied to normalized features before softmax in margin-based losses.
Distance/similarity threshold determining match acceptance; controls the FAR/FRR trade-off.
Architecture and depth of the feature-extraction network (e.g. ResNet-50, ResNet-100).
Computational complexity
Time complexity: O(N ยท D). Space complexity: O(N ยท D).
Execution paradigm
Embedding extraction is a dense CNN forward pass; all paths are active for every input.
Parallelism
Embedding extraction for many faces and gallery comparisons are fully parallelizable (batching, matrix multiplications).
Hardware requirements
Training and inference of deep CNNs benefit from parallel matrix multiplications on GPU tensor cores.
Single-face inference and embedding comparisons are feasible on CPU at small scale.