Detect and Blur Faces Programmatically
Join the DZone community and get the full member experience.Join For Free
In this post, you will learn how modern face detection algorithms work under the hood, the privacy concerns related to the use of such technology, and finally, how to make use of the PixLab API to detect faces at first, extract their coordinates and finally apply a blur filter for each extracted face (i.e. bounding boxes). Let's dive in!
Face detection has been a solved problem since the early 2000s, but it still faces a few challenges, including detecting tiny, non-frontal faces at real-time on cheap CPUs on low-end mobile/IoT devices.
The most widely used technique is a combination of Histogram of Oriented Gradients (HOG for short) and Support Vector Machine (SVM) that achieve mediocre to relatively good detection ratios given a good quality image.
PixLab, on the other side, developed a brand new architecture targeting single class object detection and suitable for face detection. This detector is based on an architecture named RealNets and uses a set of decision tress organized as a classification cascade that works at Real-time on the CPU of cheap Android devices and easily outperform the HOG/SVM combination.
The basic idea behind the RealNet face detector algorithm is as follows:
- Scan the input image/frame with a cascade of binary classifiers at all reasonable positions and scales.
- A region of the target image/frame is classified as an object of interest if it successfully passes all the members of the cascade. Each binary classifier consists of an ensemble of decision trees with pixel intensity comparisons as binary tests in their internal nodes. This enables the detector to process image regions at very high speed.
- This implementation is based on the excellent paper: Object Detection with Pixel Intensity Comparisons Organized in Decision Trees and the resulting code base is integrated and freely available within the SOD SDK.
Photo sharing on social networks can cause privacy issues. Face blurring is one strategy to increase privacy while still allowing users to share photos. Users experience ratings for face blurring were positive, indicating blurring may be an acceptable way to modify photos from the user perspective.
With privacy regulations changing all the time around the globe and the recent GDPR that came into effect in the EU, it is recommended to comply with all regulations per default by adding/ integrating face blurring into your services.
The only issue with this approach is that if the viewer has some knowledge about the user, they may still identify them, even with the blur. For instance, they may use other factors such as clothes, body shape, and skin color. However, since we've just given a basic application of the API, as a developer, you can make it more complex by blurring other features that can reveal the identity of the person in the photo. In the next section, we'll make use of the Image Blur API by PixLab to apply such filter for each detected face.
PixLab is an ML focused, SaaS platform offering Machine Vision & Media Processing APIs for developers via a straightforward Web or Offline SDKs. PixLab API feature set includes but not not limited to:
- Passports & ID Cards document scanning using state-of-the-art Machine Learning models via straightforward HTTP Rest API as shown in this Python & PHP gists and publicly available via the /docscan API endpoint.
- Board of face analysis API endpoints including face detection, landmarks extraction, facial recognition, content moderation and many more.
- On the fly image encryption, conversion, compression and full support for HTTP/2 and HTTP/3(QUIC) within the release of the PixLab API 1.9.72.
- Proxy for AWS S3 and other cloud storage providers.
- Over 130 Machine Vision & Media Processing API endpoints.
Given an input image with some nice faces:
Detect faces at first and apply a blur filter for each extracted region (i.e. face coordinates). The final result should look like the following:
The result above was obtained via the following Pyhton gist:
Python Gist Source Code: https://github.com/symisc/pixlab/blob/master/python/blur_human_faces.py
The same logic using PHP now:
PHP Gist Source Code: https://github.com/symisc/pixlab/blob/master/PHP/blur_human_faces.php
Regardless of the underlying programming language, the logic is always same. We made a simple HTTP GET request with the input image URL as a sole parameter. Most PixLab endpoints support multiple HTTP methods so you can easily switch to POST based requests if you want to upload your images & videos directly from your mobile or web app for analysis. Back to our sample, only two API endpoints are needed for our face blurring task:
- facedetect is the analysis endpoint that is called first. It outputs the rectangle coordinates for each detected human face in a given image or video frame. Each returned coordinates includes the face width, height and the X & Y position on the input image or video frame. We will use these information to apply a blur filter on each target region later. You can find out more information about the facedetect endpoint here.
- mogrify is called later after we obtain the rectangle coordinates for all the detected faces. In which case, we simply pass these coordinates to mogrify untouched and the target regions of the image are blurred. When done, mogrify returns a direct link to the blurred picture stored in PixLab CDN or your own S3 bucket if you already connected your AWS S3 bucket from the PixLab Dashboard. Note that mogrify can return the raw blurred image directly instead of cloud storage by setting the blob parameter this endpoint takes to true. Finally, The mogrify endpoint is documented at pixlab.io/cmd?id=mogrify.
Surprisingly, blurring faces automatically is straightforward for the average web developer or site administrator who may lack technical machine learning skills thanks to open computer vision technologies such the one provided by PixLab.
With privacy regulations changing all the time around the globe and the recent GDPR that came into effect in the EU, it is recommended to comply with all regulations per default by adding/integrating face blurring into your services. Find out more code samples on https://github.com/symisc/pixlab and https://github.com/symisc/sod if your are C/C++ developer for an offline SDK.
Opinions expressed by DZone contributors are their own.