Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Microsoft Cognitive Services: Face Identification

DZone's Guide to

Microsoft Cognitive Services: Face Identification

In this post, we take a look at how you can take advantage of facial identification using Microsoft's Cognitive Services! Sound interesting? Read on to find out more!

· AI Zone ·
Free Resource

EdgeVerve’s Business Applications built on AI platform Infosys Nia™ enables your enterprise to manage specific business areas and make the move from a deterministic to cognitive approach.

In today's cognitive services post, things are going to get a bit more interesting. We're moving from face detection to face identification. The difference is that we're not only going to detect that there is a face (or more faces) present on a photo but we're also going to actually identify the person that face belongs to. But to do that, we need to teach the AI about people we'd like to keep track of. Even a computer can't identify someone it has never "seen" and has no information of how they look like.

The Face API identification works on a principle of groups. You create a group of people and attach one or more faces to each group member to finally be able to find out if the face on your new photo belongs to any member of that group. (The alternative to groups are face lists, but in I'll stick with groups for now.)

The Face API supports everything you need for managing groups, people, and their faces. Here I'm expanding my Universal Windows demo application I've started building in my previous post.

Creating a person group with C# SDK is simple:

await client.CreatePersonGroupAsync(Guid.NewGuid().ToString(), "My family");

The CreatePersonGroupAsync method takes a group ID for the first parameter (easiest to use is to provide a GUID if you don't have other preferences or requirements), while the second name is a friendly name of the group that can be displayed throughout your app. There's a third (optional) parameter that takes any custom data you want to be associated with the group.

Once you've created one or more group, you can retrieve them using the ListPersonGroupsAsync method:

var personGroups = await client.ListPersonGroupsAsync();

You can start adding people to your group by calling the CreatePersonAsync, which is very similar to the above CreatePersonGroupsAsync:

var result = await client.CreatePersonAsync(personGroupId, "Andrej");

The first parameter is the same personGroupId (GUID) I've used with the above method and identifies the person group. The second parameter is the name of the person you're adding. Again, there's the third parameter for optional user data if you want some additional data to associate with that person. The return result object contains a GUID of added person.

And again, you can now list all the persons in a particular group by calling the ListPersonsAsync method:

var persons = await client.ListPersonsAsync(personGroupId);

A quick note here: both ListPersonGroupsAsync and ListPersonsAsync support paging to limit the returned result set.

Once you've added a few persons in a person group, it's time to give those people faces.

Prepare a few photos of each person and start adding their faces. It's easier to use photos with a single person to avoid one extra step of selecting particular face on the photo to be associated with a person. If only one face is detected on a photo, that one face will be added to the selected person.

var file = await fileOpenPicker.PickSingleFileAsync();
using (var stream = await file.OpenStreamForReadAsync())
{
    var result = await client.AddPersonFaceAsync(personGroupId, personId, stream);
}

It takes just a personGroupId, personId, and a photo file stream for the AddPersonFaceAsync method to add a face to a person (personId) in a person group (personGroupId). There are two more parameters, though. userData is again used for providing additional data to that face, while the last parameter, targetFace, takes a rectangle with pixel coordinates on the photo that bounds the face you want to add.

Also, instead of uploading a photo stream you can use a method overload taking a valid URL that returns a photo containing a person's face. The returned result of the above method will return the ID of persisted face that was just associated with a person.

To check how many faces are associated with a specific person, simply call the GetPersonAsync method:

var person = await client.GetPersonAsync(personGroupId, personId);

The returned person object will contain person's ID, name, user data and an array of persisted faces' IDs.

I've found that adding around 3 faces for a person is good enough for successfully identifying people in various conditions. However, I'd recommend adding faces in different conditions for improved accuracy (summer/winter, different hair styles, lighting conditions, etc.) Also, I believe adding a few faces every now and then would help in keeping the data in sync with the latest looks (like when kids are growing up).

Training

Now that we have at least one group with a few persons in it and every person is associated with a few faces, it's time to train that group.

await client.TrainPersonGroupAsync(personGroupId);

Simply call the TrainPersonGroupAsync method with the group ID to start the training process. How much it takes depends on how many persons are in the group and the number of faces, but for a small(er) amounts it usually takes a few seconds. To check the training status, call the GetPersonGroupTrainingStatusAsync  method:

var status = await client.GetPersonGroupTrainingStatusAsync(personGroupId);

The returned status includes an actual field 'status' that indicates the training status: not started, running, succeeded, and failed. You'll be most interested in succeeded and failed statuses. When you get succeeded, it means your data is trained and ready to use. In the case of "failed," something went wrong and you should check another field returned with status — the message field should report what went wrong.

Face Identification

Finally, with everything in place, we get to the fun part: identifying faces.

Face identification is a two-way process. First, you need to call the Face API to detect faces on your photo, like in. This call will return detected face's ID (or more, if multiple faces were detected). Using that ID you need to call the actual identification API to check if that face matches any of persisted faces in the particular group.    

var file = await fileOpenPicker.PickSingleFileAsync();
Face[] faces;
using (var stream = await file.OpenStreamForReadAsync())
{
    faces = await client.DetectAsync(stream);
}
var faceIds = faces.Select(i => i.FaceId).ToArray();
var identifyResults = await client.IdentifyAsync(personGroupId, faceIds);
foreach (var identifyResult in identifyResults)
{
    var candidate = identifyResult.Candidates.FirstOrDefault();
    if (candidate != null)
    {
        var person = await client.GetPersonAsync(personGroupId, candidate.PersonId);
        Console.WriteLine($"{person.Name} was identified (with {candidate.Confidence) confidence!");
    }
}

In the above code snippet, three API methods are marked bold: DetectAsync detects faces in the photo (see the previous post for more info). It will return face detected face IDs we need for the next call (Note: face IDs are stored on servers for 24 hours only, after that they will no longer be available). Taking those IDs, we call the IdentifyAsync method, also providing the personGroupId. The Face API service will then take provided face IDs and compare those faces with all the faces in the group to return results. The results contain an array of candidates for each face match; having a candidate doesn't necessarily mean we got a perfect match! We can check candidate's Confidence property that returns the match confidence score. The higher it is, more we can trust the resulting match. To finally get to the name of the person identified, we call the GetPersonAsync method with the identified person's ID.

That's it for person, groups, and faces management and basic face identification. I'll get to the more practical examples of face identification in the next posts. 

Adopting a digital strategy is just the beginning. For enterprise-wide digital transformation to truly take effect, you need an infrastructure that’s #BuiltOnAI. Click here to learn more.

Topics:
face api ,microsoft ,cognitive computing ,c# ,ai ,tutorial

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}