DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Securing Your Software Supply Chain with JFrog and Azure
Register Today

Trending

  • The Dark Side of DevSecOps and Why We Need Governance Engineering
  • The Role of Automation in Streamlining DevOps Processes
  • Grow Your Skills With Low-Code Automation Tools
  • Revolutionize JSON Parsing in Java With Manifold

Trending

  • The Dark Side of DevSecOps and Why We Need Governance Engineering
  • The Role of Automation in Streamlining DevOps Processes
  • Grow Your Skills With Low-Code Automation Tools
  • Revolutionize JSON Parsing in Java With Manifold
  1. DZone
  2. Software Design and Architecture
  3. Security
  4. Working With Face Recognition: Misty as a Security Guard: Part 1

Working With Face Recognition: Misty as a Security Guard: Part 1

Misty’s got a new job: security guard.

Johnathan Ortiz-Sonnen user avatar by
Johnathan Ortiz-Sonnen
·
Updated Apr. 30, 19 · Tutorial
Like (4)
Save
Tweet
Share
11.07K Views

Join the DZone community and get the full member experience.

Join For Free

In the security guard skill, Misty sounds the alarm when she sees an intruder in the room.

Misty’s got a new job: security guard.

Misty loves to be helpful, and this job combines a few of her different capabilities into a single working skill. When the skill runs, Misty activates face recognition and scans the environment for strangers. If Misty sees an unknown face, she calls out to third-party APIs and prints a picture of the intruder. Face recognition, external HTTP calls, and image capture are the core capabilities that Misty uses for this job.

In this series, we’ll walk through these capabilities one at a time so you can build a security guard skill yourself. We’ll start by understanding how face recognition works on Misty and in your code.

Misty watches as a developer codes her a new skill.
That look on Misty’s face? “More skills, please!”

But first — what’s a skill?

When we talk about skills, we mean your JavaScript code, running locally on Misty. Local skills can be as simple as two files — a code file with the JavaScript Misty executes when the skill runs and a .json file with meta information about the skill. In local skills, events and callback functions handle most of the data from Misty’s sensors and processes. Misty’s JavaScript API provides methods for subscribing to event data, and you define the callbacks for handling this data in your skill code.

With that out of the way, let’s get to the fun stuff.

Understanding Face Recognition

Face recognition has changed how we interact with computers and people alike. It enables applications that range from absurd — like animating rainbow vomit in chat apps — to actually very useful — like searching your phone for pictures of a specific friend. As the tech has improved, face recognition has taken on critical roles in more parts of everyday life. Think about how many times you’ve unlocked your phone today by looking at the camera, or how face recognition has emerged (for better or worse) in video surveillance networks around the world.

Face recognition doesn’t just feel magical, it comes in handy for all kinds of robot jobs. Whether you’re coding Misty to deliver a package to the right recipient or to express joy when your spouse gets home, face recognition hugely multiplies her usefulness as a human companion.

Misty detects and recognizes a developer's face.

Misty uses a module based on the Snapdragon Neural Processing Engine to detect and recognize faces captured by the camera in her visor. This process runs locally on each Misty robot, so she can use face recognition even without a connection to the internet. Check out this video for a much deeper dive into Misty’s computer vision capabilities.

Registering for Face Recognition Events

Before Misty can recognize people, she must be “trained” on their faces. To do this, you can use either her API or Misty’s API Explorer. The latter allows you to get started quickly if you don’t want to handle this in your own application. Once Misty knows your face, starting face recognition in a skill is as easy as:

misty.StartFaceRecognition();

This command activates Misty’s face recognition process so that Misty starts evaluating whether she recognizes the faces she sees. For your skill, however, that’s not quite enough. To use face recognition data in your code, you must register for the FaceRecognition event type. This is the event Misty sends whenever she sees a face, whether she recognizes it or not.

In the snippet below, the user-defined function registerFaceRec() does this registration. It starts with a call to misty.AddPropertyTest() to ensure FaceRecognition events only trigger the callback function when a face is actually in view, and not when registration or other types of messages are sent. This means there should be a valid string value for PersonName, be it the name for a trained face or “unknown person”.

We then call misty.RegisterEvent() to register to receive those specific FaceRecognition events.

function registerFaceRec(){
misty.AddPropertyTest("FaceRec", "PersonName", "exists", "", "string");
misty.RegisterEvent("FaceRec", "FaceRecognition", 1000, true);
}

registerFaceRec();

Creating a Face Recognition Callback

The next step is to define the callback for handling the face recognition event data. By default, callback functions have the same name you specified for the event, prefixed with an underscore. So, in this case, the callback for handling FaceRec data would be called _FaceRec().

When Misty detects but doesn’t recognize a face, the value of PersonName in the FaceRec data object is “unknown person”. Good security guards get suspicious when they see strangers where they don’t belong. For Misty to react appropriately, you might write a callback that looks like this:

function _FaceRec(data) {
// Check if the FaceRec event was triggered by a stranger
if (data.PropertyTestResults[0].PropertyValue == "unknown person"){
// Misty doesn't recognize this person. Sound the alarm!
} 
else {
// Misty knows this person. Do something else.
 }
}

If the face recognition data indicates an unknown person, we can have Misty perform security-related responses in the callback’s if block. If Misty does know the person, we can use the else block to have her behave warmly to her known companion.

CP gives the inside scoop on how face recognition works in a security guard skill.

The job at hand determines the complexity of your skill’s architecture. In the code for the security guard skill, the _FaceRec() callback triggers some fairly nuanced behavior; catching an intruder kicks off the execution of the other abilities we’ll discuss in this series. When you look at the full code file, you’ll also see helper functions, logic to handle edge cases (i.e. Misty won’t react to faces she detects unless they’re within a certain distance), and a loop that keeps the skill running indefinitely.

What’s Next?

In the next post, we’ll look at how Misty sends data to third-party cloud services. In the meantime, you can check out the full demo here. Head over to GitHub to see the skill code for yourself, or check out what else Misty developers are working in the Misty Community forum.

Guard (information security) security

Published at DZone with permission of Johnathan Ortiz-Sonnen. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • The Dark Side of DevSecOps and Why We Need Governance Engineering
  • The Role of Automation in Streamlining DevOps Processes
  • Grow Your Skills With Low-Code Automation Tools
  • Revolutionize JSON Parsing in Java With Manifold

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: