DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
  1. DZone
  2. Coding
  3. Languages
  4. Face Detection with getUserMedia

Face Detection with getUserMedia

Raymond Camden user avatar by
Raymond Camden
·
Apr. 16, 12 · Interview
Like (0)
Save
Tweet
Share
8.89K Views

Join the DZone community and get the full member experience.

Join For Free

there are quite a few interesting apis evolving in the "modern web", but not all of them are going to be things you would use in most projects. i've been very public about my feelings concerning canvas for example. great for games and charting - but not much else. that doesn't make it a bad feature. it just makes it one i won't use terribly often. whenever i read about some new cool feature being developed, my mind starts trying to figure out what they could be used for in a practical sense. obviously what's practical to you may not be practical to me, but figuring out how i would actually use a feature is part of how i learn it.

one such feature is getusermedia ( w3c spec ). this is a javascript api that gives you access to (with permission) the user's web cam and microphone. getusermedia is currently supported in opera and chrome (i believe it is in version 18 now, but you may need to grab canary. you also need to enable it. instructions on that here .) once you get past actually enabling it, the api is rather simple. here's a quick request for access:

//a video tag
var video = document.getelementbyid('monitor');

//request it
navigator.webkitgetusermedia('video', gotstream, nostream);

function gotstream(stream) {

	video.src = webkiturl.createobjecturl(stream);
	video.onerror = function () {
		stream.stop();
		streamerror();
	};
}

function nostream() {
	document.getelementbyid('errormessage').textcontent = 'no camera available.';
}


function streamerror() {
	document.getelementbyid('errormessage').textcontent = 'camera error.';
}

the first argument to getusermedia is the type. according to the spec, this is supposed to be an object where you enable audio, video, or both, like so: {audio:true, video:true}. however in my testing, passing a string, "video", worked fine. the demo you will be seeing is based on another demo so that line possibly came from an earlier build that still works with chrome. the second and third arguments are your success and failure callbacks respectively.

you can see in the gist where the success handler assigns the video stream to an html5 video tag. what's cool then is that once you have that running you can use the canvas api to take pictures. for a demo of this, check out greg miernicki's demo:

http://miernicki.com/cam.html

if this demo doesn't work for you - then stop - and try following the instructions again to enable support. (although i plan on sharing a few screen shots so if you just want to keep reading, that's fine too.)

based on greg's demo, it occurred to me that there is something cool we can do with pictures of our web cams. (cue the dirty jokes.) i remembered that face.com had a very cool api for parsing pictures for faces. (i blogged a coldfusion example back in november.) i wondered then if we could combine greg's demo with the face.com api to do some basic facial recognition.

turns out there are a few significant issues with this. first - while face.com has a nice rest api, how would we use it from a javascript application? secondly - face.com requires you to either upload a picture or give it a url. i know i could send a canvas picture to a server and have my backend upload it to face.com, but is there a way to bypass the server and send the picture right to the api?

the first issue actually turned out to be a non-issue. face.com implements cors (cross-origin resource sharing). cors basically allows a server to expose itself to ajax calls from documents on other domains. it's a great feature and i hope more services enable it.

the more complex issue then was taking the canvas data and sending it to face.com. how can i fake a file upload? turns out there's another cool new trick - formdata. fellow coldfusion blogger sagar ganatra has an excellent blog entry on the topic. here's how i used it:

function snapshot() {
	$("#result").html("<p><i>working hard for the money...</i></p>");

	canvas.width = video.videowidth;
	canvas.height = video.videoheight;
	canvas.getcontext('2d').drawimage(video, 0, 0);

	var data = canvas.todataurl('image/jpeg', 1.0);
	newblob = datauritoblob(data);

	var formdata = new formdata();
	formdata.append("api_key", facekey);
	formdata.append("api_secret", facesecret);
	formdata.append("filename","temp.jpg");
   
	formdata.append("file",newblob); 

	$.ajax({
	   url: 'http://api.face.com/faces/detect.json?attributes=age_est,gender,mood,smiling,glasses',
   	   data: formdata,
	   cache: false,
	   contenttype: false,
	   processdata: false,
	   datatype:"json",
	   type: 'post',
	   success: function (data) {
	       	handleresult(data.photos[0]);
	   }

	});    
}

let's look at this line by line. first off - i need to get the binary data from the canvas object. there's a few ways of doing this, but i wanted a binary blob specifically. notice the datauritoblob method. this comes from a stackoverflow post i found a few weeks back.

i create a new formdata object and then simply begin setting my values. you can see i pass in a few api requirements but the crucial parts are the filename and file object itself.

below that you can see the simple jquery ajax call. face.com has a variety of options, but i basically just asked it to return an estimated age, gender, mood, and whether or not the person was smiling and wearing glasses. that's it. i get a nice json packet back and format it.

now obviously no api is perfect. i've had different levels of results from using the api. sometimes it's pretty damn accurate and sometimes it isn't. overall though it's pretty cool. here are some scary pictures of yours truly testing it out.

ok, ready to test it yourself? just click the demo button below. for the entire source, just view source! this is 100% client-side code.

for another look at getusermedia, check out these examples:

  • it's curtains for marital strife thanks to getusermedia
  • testing webrtc on chrome
  • bleeding edge html5, webrtc & device access
  • capturing audio & video in html5

IT Object (computer science) Upload Web Service Data (computing) JavaScript application

Published at DZone with permission of Raymond Camden, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Required Knowledge To Pass AWS Certified Solutions Architect — Professional Exam
  • Front-End Troubleshooting Using OpenTelemetry
  • 5 Software Developer Competencies: How To Recognize a Good Programmer
  • Create a REST API in C# Using ChatGPT

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: