Facial Recognition and Identification in Computer Vision

Rapid development in machine learning and AI technologies have made significant progress in computer vision for object detection and recognition. Yet creating a system that can recognize humor and amusement in people is still a challenging problem for programmers, since comedy and entertainment are subjective and based on personal tastes and cultural context. Thus, it is tricky creating a system that can precisely determine what individuals find amusing or enjoyable.

However, computer vision in facial recognition and identification has made significant progress, widely employed in various applications such as security, surveillance, and biometrics. Facial recognition technology mainly identifies individuals by analyzing and comparing their facial features to a database of known faces. Facial identification involves matching a person's face to their identity. It employs algorithms to extract face characteristics like the distance between the eyes, nose curves, and the angle of the jawline to generate a distinctive facial signature.

Techniques for Facial Recognition and Identification

There are various methods for recognizing and identifying faces, but this tutorial will focus on three techniques and outline their benefits and drawbacks.

1. Template-Based Approach

Firstly, the template-based approach compares the facial features of a target face with those in a template or reference image. The template image comprises several facial features, such as the position of the eyes, nose, mouth, and the distances between them. The technique then compares the target face with the template image to determine if there is a match. Eigenfaces is one of the most common template-based methods that uses principal component analysis (PCA) to reduce the dimensionality of facial images and extract the most relevant features.

Advantages

• They are very accurate with high-quality templates
• Handles variations in lighting and facial expressions

Disadvantages

• Requires high-quality templates for accurate results
• May be computationally expensive

2. Feature-Based Approach

Secondly, the feature-based technique extracts features from the face, such as the shape of the eyes, the distance between the eyebrows, and the curvature of the lips. The approach then compares these features with those in a database of known faces to identify the target face. Examples of typical feature-based techniques include the Local Binary Pattern (LBP) and Scale-Invariant Feature Transform (SIFT).

Advantages

• Relatively fast and efficient
• Works with low-resolution images

Disadvantages

• May not work well with varying lighting conditions
• May not be robust to changes in facial expressions or poses

3. Deep Learning-Based Approach

Lastly, a deep learning-based approach uses deep neural networks to learn features directly from facial images. Networks are trained on large datasets of facial images to identify patterns and feature relevant to facial recognition. Examples of typical deep learning-based methods include Convolutional Neural Networks (CNN) and Siamese Networks.

Advantages

• Achieves high accuracy with large datasets
• Handles variations in facial expressions and poses

Disadvantages

• Requires a large amount of training data
• Computationally expensive

NOTE: There are several techniques for facial recognition and identification.

Facial Recognition and Identification Using Python and OpenCV

With the availability of powerful libraries like OpenCV and the advancements in machine learning algorithms, it has become easier than ever to develop facial recognition and identification systems using Python. OpenCV provides a rich set of functions and tools for image processing, particularly useful in facial recognition tasks such as face detection, feature extraction, and matching. It also has pre-made models such Haar Cascades classifier for face detection and VGG or ResNet to label the detected faces.

OpenCV provides various models for face labelings, such as Eigenfaces, Fisherfaces, and Local Binary Patterns Histograms (LBPH). We train these models to identify specific individuals by presenting them with images of the individuals we want to recognize. Eigenfaces uses Principal Component Analysis (PCA) to construct a face image from smaller parts, while Fisherfaces utilizes Linear Discriminant Analysis (LDA) to distinguish between faces. LBPH compares the texture of faces to known faces.

Facial Identification

Import Dependencies

Import the necessary libraries.

import numpy as np
import cv2
import matplotlib.pyplot as plt
from IPython.display import Image

from google.colab import drive
drive.mount('/content/drive')

Load Pre-Trained Models and Image

OpenCV provides a set of pre-trained `Haar Cascade classifiers` for detecting various objects, including faces. `cv2.CascadeClassifier` is a class that represents the Haar Cascade classifier. The cv2.data.haarcascades is a path to the directory containing the pre-trained models. `haarcascade_frontalface_default.xml` is the file containing the pre-trained face detection model. The model is available for free download on GitHub.

# Load the pre-trained face detection model
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

Load the input image `image1.jpg` can be an image of your choice using OpenCV's `imread()` function. It then converts the image to grayscale using OpenCV's `cvtColor()` function. The grayscale image is required for the face detection algorithm.

# Load the image and convert it to grayscale
img = cv2.imread('/content/drive/MyDrive/Face Detection/image1.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#show the image 
plt.imshow(gray)

The line below detects faces in the grayscale image using the detectMultiScale() method of the CascadeClassifier class. The detectMultiScale() method takes the following arguments:

`gray`: the grayscale image

`scaleFactor`: is a parameter specifying how much the image size is minimized at each image scale. A value of 1.3 means the image is reduced by 30% at each scale.

`minNeighbors`: is a parameter specifying how many neighbors each candidate rectangle should have to retain it, higher value results in fewer detections but with higher confidence.

`The detectMultiScale()` method returns a list of rectangles where faces are detected. Each rectangle represents a tuple of four integers `(x,y,w,h)` representing the top-left corner coordinates `(x,y)` of the rectangle and its width `w` and height `h`.

# Detect faces in the grayscale image
faces = face_cascade.detectMultiScale(gray, 1.3, 5)

Drawing Rectangle at Face Region

Let's create a loop that draws rectangles around the detected faces in the original image. The cv2.rectangle() function takes the following arguments:

`img`: the original image.    

`(x,y)`: the top-left corner coordinates of the rectangle.    

`(x+w,y+h)`: the bottom-right corner coordinates of the rectangle.    

`(255,0,0)`: represents the binary color of the rectangle drawn in RGB format. (255,0,0) represents blue.

`5`: the thickness of the rectangle border in pixels.

# Draw rectangles around the detected faces
for (x,y,w,h) in faces:

    cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),5)

#Show the image after building the rectangle.


# Display the image with the detected faces

plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))

plt.show()

# Save the detected image by name
cv2.imwrite('detected_image.jpg', img)

#Confirm if saved image exists.
detected_image=cv2.imread("detected_image.jpg", 1)
gray_channels_reversed = detected_image[:, :, ::-1]
plt.imshow(gray_channels_reversed)

Facial Recognition

We will not go into an explanation like the one above but use the above saved detected image to confirm if the model works for identification by giving different images for detection.

Import Dependencies

import numpy as np
import cv2
import matplotlib.pyplot as plt
import dlib
from imutils import face_utils
from scipy.spatial.distance import cosine
from sklearn.metrics.pairwise import cosine_similarity

Load Face Detection Model and Facial Landmark Detector

The `dlib.get_frontal_face_detector()` function returns a pre-trained face detection model capable of detecting human faces in images. For more info, check dlib.net functions. Also, download the landmark detector.

# Load the face detection model and the facial landmark detector
face_detector = dlib.get_frontal_face_detector()
landmark_detector = dlib.shape_predictor('/content/drive/MyDrive/Face Detection/shape_predictor_68_face_landmarks_GTX.dat')

# Load the detected image of 

img1 = cv2.imread('/content/drive/MyDrive/Face Detection/detected_image.jpg')
gray1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)


# Detect the face and facial landmarks of the image
rects1 = face_detector(gray1, 1)
landmarks1 = landmark_detector(gray1, rects1[0])
landmarks1 = face_utils.shape_to_np(landmarks1)

# Load another image to compare with the image loaded previously
img2 = cv2.imread('/content/drive/MyDrive/Face Detection/001_d3323f3c.jpg')
gray2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)

### Set model parameter

# Detect the face and facial landmarks of the second image

rects2 = face_detector(gray2, 1)

landmarks2 = landmark_detector(gray2, rects2[0])

landmarks2 = face_utils.shape_to_np(landmarks2)

# Extract the features of the two images by computing the Euclidean distances between facial landmarks

features1 = np.zeros(68)

features2 = np.zeros(68)

for i in range(68):

    features1[i] = np.linalg.norm(landmarks1[i] - landmarks1[0])

    features2[i] = np.linalg.norm(landmarks2[i] - landmarks2[0])

#### Train on similarities

# Compute the cosine similarity between the two feature vectors

score = cosine(features1, features2)

# Check if the score is below a certain threshold (e.g. 0.5) to determine if the two faces match

if score < 0.5:

    print("The model did not recognize Joseph in one or both of the images..")

else:

    print("The model recognized Joseph in both images.")

Conclusion

In this tutorial, you have learned facial recognition and identification using Python OpenCV, how it works, its uses, and how it can be implemented. The code for this project can be accessed from the GitHub Repo.