Over a million developers have joined DZone.

Can You Automate Prejudice?

· Big Data Zone

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

Big data is, well, big. Its application is creeping into more and more of our daily lives as companies look to utilize the masses of data generated by us to improve, and in many instances, automate their dealings with us.

Recruitment is one area that has certainly taken this approach. Companies are increasingly making use of algorithms to sort and sift through job candidates, weeding the list down significantly before a human being even enters the fray.

A British university has a salient lesson to teach us, however, on how such an approach can go rather terribly wrong. Staff at St George’s Hospital Medical School wanted to improve their admissions process, so wrote an algorithm to automate the first round.  The algorithm was based upon historical patterns, so they used data from candidates historically rejected to form an identikit of people they should filter out in the first round.

Whilst there are clear concerns about people being chosen by computers rather than people, a more worrying trend was spotted. It was found that the admissions data used was biased against both female applicants and those with non-European sounding names.

It’s a salient lesson at a time when the likes of LinkedIn are using similar algorithms to make recommendations to recruiters on candidates they may wish to consider based upon past recruitment choices by that consultant.

Whilst there may be practical reasons to hand over part of the recruitment process to computers, this will hopefully provide the reminder that this most human of endeavors still requires the human touch.


Hortonworks Sandbox is a personal, portable Apache Hadoop® environment that comes with dozens of interactive Hadoop and it's ecosystem tutorials and the most exciting developments from the latest HDP distribution, brought to you in partnership with Hortonworks.

Topics:

Published at DZone with permission of Adi Gaskell, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}