Over a million developers have joined DZone.

How Algorithms Rule the World

DZone's Guide to

How Algorithms Rule the World

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

In a brainstorming meeting not too long ago, I asked my colleagues what I thought would be an easy question: What successful applications use artificial intelligence and machine learning? While I personally didn't know of many, I figured that collectively we'd be able to rattle off a dozen or so answers.

Instead, we had trouble thinking of more than a handful (with Pandora and Netflix leading the way).

The fact is, there are many successful products that use these types of advanced algorithms, but few companies are eager to share the "secret sauce" that drives decision-making engines. It is no wonder, then, that we were at a loss to name many.

This last week I've been reading Chris Steiner's excellent book Automate This: How Algorithms Came to Rule Our World. Chris's thesis is that these algorithms may not be pasted across billboards, but they are in fact hidden all around us, from Amazon, Pandora, and Netflix to the trading floors of Wall Street and Chicago.

Automate This bounces back and forth between academia, Silicon Valley, and the trading floors as Chris traces the lineage of algorithmic decision making. Traders rely on algorithms to predict the markets, and often to even make the trades for them. The music industry uses computers to analyze singles and tag which ones are likely to be pop super-hits. Pharmacies can use automated prescription fillers to reduce error rates by vast amounts. Customer support at large companies uses voice analysis to guide the customer service rep's interactions -- and in some cases even to select which rep will best meet a customer's needs. Even organ donation pools now benefit from sophisticated matching algorithms.

Steiner rightly raises some concerns about safety, ethics, and social ramifications of over-reliance upon these algorithms, but even so he does this while pointing out reliable these algorithms are in cases where (as behavioral economists so frequently point out) human reasoning is less than ideal.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}