DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. AI/ML
  4. Use Machine Learning to Find Waldo

Use Machine Learning to Find Waldo

You can use artificial intelligence and image recognition technology to find all the Waldos. Here is one way of going about it!

Minh Tran user avatar by
Minh Tran
·
Jul. 31, 17 · Tutorial
Like (7)
Save
Tweet
Share
8.45K Views

Join the DZone community and get the full member experience.

Join For Free

At Clarifai, we have an internal Hack Day every month where everyone works on a pet project they don't normally have time for. My Hack Day project this month was to attempt to answer a question that bored kids everywhere have asked when they found themselves stuck indoors, be it from either from a rainy day, a doctor's visit, or a cross country road trip in the back of a minivan: Where's Waldo?

With scenes filled with animals, aliens, mermaids, and people of all sorts (including several Waldo imposters or "friends" as they call them), I wanted to see just how well our custom training model could perform on such a challenging and popular data set. Here's how I did it!

waldo1

Step 1: Find the Data

Luckily for me, some kind and curious soul had already created and shared a Where's Waldo? dataset online, where he took 19 Where's Waldo maps, split them into grids/tiles, then labeled the tiles accordingly ("waldo" or "not waldo").

I decided to follow that method and do the same with other Waldo maps not included in his set. Maps were split up in several different grids (4x5, 4x6, 5x8, etc.) to increase the sample sizes. These tiles were then uploaded into our app and labeled accordingly.

whereswaldo_begin

Step 2: Train, Test, More Data

After the initial training, I tested the model against a map not in the training set, featuring Waldo on a moon colony, naturally. Against the entire map, our model found... nothing. Well... a 1% chance of Waldo being present. Not an unexpected result, since so much of the image simply isn't Waldo — that's what makes playing these so fun/frustrating.

I decided to test it again using the same grid system as the training. Going from tile to tile, results started making more sense. 1-3% chance of Waldo on relatively empty tiles. 10-20% chance on busier ones. 30-40% chance on Waldo imposters (shakes fist). Then, a tile with 50% chance of Waldo! I scanned the tile anxiously. Boom. Waldo, hiding behind a crowd of people on the second floor of a biodome.

Step 3: More Data, Fewer Imposters

My next goal was to lower the percentages of the Waldo look-alikes being flagged as Waldo. I went back to the original maps and manually cropped out several (somewhere between 500-1,000) people; some look-alikes, others with similar color schemes, others just random people. All to tell the model who wasn't Waldo.

whereswaldo_find

Results

Higher percentages on Waldo positive tiles, lower percentages on Waldo negative tiles, fewer false-positives on lookalikes.

Overall, though, the numbers still aren't great. The busier tiles still throw the model for a loop, Waldo predictions still aren't high enough, and false-positives are still present. Oddly enough, it isn't the characters who would normally trick human eyes (the lookalikes, characters wearing stripes or glasses, etc.) that are throwing the false-positives, but something else that I haven't been able to discern yet.

As I revisit this project, I believe my next steps would be to find/increase the Waldo-positive examples to try and counteract the implicit unbalanced nature of the data. That's the beauty of machine learning — the more examples you show, the better the results.

Machine learning

Published at DZone with permission of , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • How To Use Terraform to Provision an AWS EC2 Instance
  • Choosing the Best Cloud Provider for Hosting DevOps Tools
  • Using JSON Web Encryption (JWE)
  • Easy Smart Contract Debugging With Truffle’s Console.log

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: