We’re going to be hearing more in the coming months and years about machine learning and how it works alongside Big Data as a way to ‘turn the corner’ on the challenges of data’s volume, velocity and variety. Often called optimization or normalization, machine learning allows a computer to ‘learn’ a better model for solving a problem by exposure to known data sets, with the expectation that it’s processing power can find trends and occurrences in new and unseen datasets. The more involved humans are in the process, the more machine learning is called ‘supervised’ and the less, ‘unsupervised.’
It’s advantages are simple:
- It performs more complex analysis than humans in many cases
- It can be faster and more real-time than batch-driven analysis cycles
- Through automation, it can cycle through answers on the fly, while new data is being generated
Big Data Republic offers a good, succinct definition of machine learning in the form of a video store example:
By using machine learning technology, customers to Tobias’s online movie store get a more personalized, evolving service. Based on pages and products viewed, a customer to the site is presented with potential films he or she might like to purchase. This is based on the machine learning engine spotting correlations in the data of customers with similar demographics who have viewed similar pages, and recommending potential purchases from their purchase history.
As this is an automated system that “learns,” Tobias doesn’t need to be constantly tweaking the algorithms, and the machine learning tool continues to learn, so when a new natural purchasing trend emerges among customers, it makes recommendations based on having recognized these new patterns.
As the world gets faster, the need to find faster solutions despite the growth and complexity of data leads us toward machine learning. It offers the opportunity, depending on the use case and desired outcome, to reduce the amount of analysis done by expensive and scarce data scientists. Expect to hear more.