How Amazon Uses Its Own Cloud to Process Vast, Multidimensional Datasets

A smart TV that knows which shows to record, an espresso coffee machine that raises an alert when it requires maintenance, a refrigerator that alerts you when the milk is out of stock - are all powered by data analytics. With the rapidly increasing demand to predict the future of markets and consumer behaviour - predictive analytics has become a necessity for retailers like Walmart, Amazon, eBay and Nordstrom.

According to a new report from EKN Research, 80% of e-commerce giants say that they lag behind Amazon in analytics maturity. Jeff Bezos and his team at Amazon have revolutionized online shopping for consumers over the last 17 years, with a revenue of $2 billion per year (including its supreme e-commerce and cloud computing services) to show for it. Below is a case study of how Amazon leverages Big Data analysis for its business. A similar case study on Walmart can be found here. Read the Walmart Big Data Analytics Case Study.

Seattle-based e-commerce giant Amazon, is leveraging big data on its 200 million customer accounts by hosting their 1,000,000,000 GB of data on more than 1,400,000 servers to increase sales through predictive analytics. Data is the lifeblood of Amazon, which it uses to ensure that it is doing something special for its customers through personalized recommendations, price optimization, targeted marketing and more. Big data analytics is the magic wand for Amazon that helps form loyal customer relationships in a fierce competitive world by leveraging personal data. The ability to use data and the right to innovate is the building block for Amazon’s debut into Fortune 500 global retail empire.

“Data is Power” is the success mantra at Amazon. Just look at your Amazon homepage, it is never the same. Amazon tracks everything you do at the e-commerce website and app - to collect as much data as it can. A look at the “Account” section on Amazon will give an idea about the robust account management they have, to collect data from its customers. With various sections on the homepage like – “Inspired by Your Wish List,” “Recommendations for You,” “Inspired by Your Browsing History,” “Related to Items You Have Viewed,” “Customers Who Bought This Item Also Bought,” Amazon is continuously tracking what their customers do - to provide them a personalized preeminent shopping experience.

For example below we see that if you select to buy a hard drive in Amazon.com you will also be shown products that are frequently bought with the hard drive and the other choices besides the hard drive that you would like to purchase.

Use of Hadoop Technology at Amazon

Amazon has expanded its usage of big data through various tools in the cloud for data storage, data collection, data processing, data sharing and data collaboration. Amazon’s Elastic MapReduce helps the retailer effectively manage and use the analytics platform, built on top of Hadoop framework. The product catalogue data of about 1.5 billion products in the retail store, spread across 200 fulfilment centres across the globe and stored in Amazon S3 interface, receives close to 50 million updates every week. The product catalog of data in S3 is crunched every 30 minutes and sent back to various data warehouses.

Personalized Recommendations at Amazon

Amazon is winning the heart of its customers by analysing big data to find what customers actually buy, what they are searching for and what actually they need.

Amazon generates an additional 10%-30% revenue in response to the recommended suggestions it offers to its customers. Amazon is the pioneer in mining the big data to provide personalization to entice customers with a curated shopping experience. With more than 2 million sellers across 10 countries serving close to 200 million customers - Amazon provides personalized recommendations to its customers through its super innovative data-driven technology.

Amazon thinks of its customers in two ways-

• Amazon-buying customers
• Sellers on the Amazon Marketplace

Amazon recommendation algorithms incorporate various factors like- purchase history, browsing history, impact of friends, trends for a particular product, social media mentions on the popularity of a product, purchases made by customers with similar purchase history, etc. before they recommend a product to a customer. Amazon is always on the verge of refining its recommendation algorithms with a quest to better serve its customers.

Sellers on e-commerce marketplace have several questions like –

Amazon boasts of great success with its sellers as it provides recommendations on inventory stock to sellers, provides suggestions on new product choices they should add to their inventory, which is the best delivery mode for particular products to meet on-demand deliveries, etc. On an average, a seller in Amazon might have 100+ suggestions in their queue on various product categories.

The most challenging and powerful criteria for satisfying customers is to manage the inventory better. One of the popular suggestion Amazon provides to its sellers is about the inventory being out of stock. The recommendations algorithms analyze how much a particular seller is selling and how much inventory does a seller have in stock. Based on all these factors the recommendation algorithms at Amazon makes suggestions to sellers on the forward-looking demand for their product so that the sellers can add more inventory to their marketplace in Amazon, making it a WIN-WIN strategy.

Dynamic Price Optimization at Amazon

Price optimization is an important factor in the retail market as retailers struggle to offer best price on each product. Price management is monitored closely at Amazon to attract buyers, win the battle against other competitors and grow the business on profit margins.

Dynamic pricing helped Amazon boost profits by 25% on an average and they stay competitive by monitoring the price 24x7, 365 days. Amazon increased its sales by 27.2% from 2012 to 2013 and made an entry into the list of top 10 retailers in US for the first time. Product pricing strategy at Amazon supports real-time pricing by considering data from various sources like customer activity on the website, available inventory of a product, competitor pricing for a product, order history, preferences set for a product, expected margin on the product, and more. Amazon changes the price of its products every 10 minutes.

An average shopper always finds that Amazon’s product prices are the lowest among all their competitors on the web - thanks to their Dynamic Pricing Strategy. Amazon’s dynamic pricing algorithms tweak each product’s price several times in an hour to make the best use of human psychology about price perception. Amazon provides huge discounts on best-selling products whilst making better profits on slightly less popular ones.

For example, Amazon prices a best-selling Smartphone 25% below its competitors’ price whereas a less popular model of the smartphone is 10% more expensive than its cost in the competitors’ website.

According to an analysis report by Boomerang Commerce -"Amazon may not actually be the lowest-priced seller of a particular product in any given season. But its consistently low prices on the highest viewed and best-selling items drive a perception among consumers that Amazon has the best prices overall - even better than Walmart."

There are several satisfied customers on Amazon, who have scored a great deal because of dynamic pricing and the disappointed ones too for not having made a purchase on Amazon at right time. One best example that relates to Dynamic pricing is when Amazon slashed down the price of Star Wars Blu-ray set to $70 and later a week later the price was increased to $134.Customers who bought the box were excited and happy  for having bagged the best deal whilst others who paid more were in a gleam of disappointment.

Supply Chain Optimization at Amazon

Amazon owes its success to enhanced shopping experience for customers but it is meaningless without effective supply chain and fulfilment capabilities. According to a survey conducted by CapGemini- 89% of US customers would go elsewhere for their next purchase if they face troubles with on-time order fulfilments. Amazon ships approximately 10 million products when compared to Walmart’s 500,000.

Amazon has real-time links with manufacturers, tracks the inventory demand based on the data to provide same day or next day delivery options to its customers. Amazon uses big data systems to select data warehouses based on the proximity of vendors balanced against the proximity of customers to cut down on the distribution costs. The big data systems helps Amazon predict the number of data warehouses needed and the capacity each warehouse should have.

Amazon uses graph theory to minimize the delivery costs by choosing optimal schedules, routes and product groupings.

Amazon Knows What You Want Before You Buy it - Anticipatory Shipping

Amazon has obtained a big take-over among its competitors by acquiring a patent for Anticipatory Shipping to make quicker deliveries. It has patented the process of shipping a product to a customer with an expectancy that the customer will order that product based on the power of predictive analytics. The patent signifies that Amazon believes, that the predictive analytics systems will become so accurate that they will be able to predict what a customer will buy and when.

There are many traditional retailers providing personalized recommendations but Amazon is trying to take it up to the next level through the patent by accurately predicting the items that a customer would like to buy. If the big data algorithms go wrong with the predictions then Amazon might have to face a tough time in bearing the logistics cost for shipping the product and returning it back to the fulfilment centres. There could be several such problems with anticipatory shipping and we just need to wait and watch on how these problems are tackled by Amazon to stay competitive.

Retailers like Walmart, Amazon, eBay, Nordstrom are making big data investments to maintain customer relationships and grow their business. It is surprising to see how retailers are making accurate predictions about customer behaviour to drive business value in the long run. Chime in your opinion in comments about other innovative data-driven approaches used by retailers to drive business value.