Amazon Web Services advertises four principles that guide their pricing strategy - pay as you go, pay less by using more, save when you reserve, and their free tier. These principles provide immense benefits and efficiencies for thousands of organizations, which has driven AWS’s stellar growth. But there are downsides as well. In this article we'll review the five principles and provide a grain of salt you should consider before hooking into the Amazon machine. Principle #1: Pay as You Go The Principle: This is the main idea behind AWS - instead of buying or building costly infrastructure, rent it. AWS is dedicated to turning your CapEx expenses into OpEx. It also provides extreme flexibility - you can order 1,000 machines for an hour and then stop them and pay only for those 1,000 machine hours. The Good: It’s why organizations started moving to the cloud. Pay per use is great because it eliminates overcapacity and wasted computing resources. The Bad: Flexibility comes at a cost. If you want to really “pay as you go”, you’ll have to settle for Amazon’s On-Demand Pricing, which becomes quite expensive even for small workloads, when used for ongoing server deployments. Truth be told, most servers don’t have huge peaks or troughs in their usage - and running them on dedicated hosting or on-prem will be a lot cheaper than on Amazon. Principle #2: Pay Less by Using More The Principle: AWS provides volume discounts. Amazon S3 and many other services offer tiered pricing, and Amazon EC2 offers volume discounts for users who spend more than $500,000 in upfront costs. Amazon also provides a plethora of services and options for most use cases, allowing you to switch to a service that meets your need at a lower cost. For example, there are several AWS backup options including the AWS Backup service and storage services like S3, Glacier, EBS, EFS, etc. Organizations can move data between these storage services to gain efficiencies. The Good: Sophisticated users of AWS can save a lot by dynamically moving workloads between services and creating economies of scale. The Bad: This principle is also one of the hidden reasons for Amazon’s enormous complexity. True - you can create a tiered storage strategy and save 90% or more in many cases. But do your engineers or IT staff know the intricacies of each data service, and have the know-how to detect the relevant events and store data selectively into different data stores? Amazon provides the tools to do all this. But it requires time and expertise which by itself costs organizations serious money. Principle #3: Save When You Reserve The Principle: At the core of AWS is its compute service, Amazon EC2. EC2 machine instances are substantially discounted (on the order of 30-50%) if you reserve an instance for 1-3 years in advance. Another option is to use “spot instances” - machine instances that happen to be available at a given time, and will be taken away from you when another user demands them. Switching loads dynamically between spot instances, and helping Amazon manage their demand, can give you even bigger discounts. The Good: Amazon provides a lot of price flexibility. You can significantly cut costs by committing to 1 years or more - it’s possible to do this selectively for some workloads, while using others on demand. The spot instances option is a creative one, which lets anyone with expertise, and the time to architect a spot instances solution, shave 60% off costs. The Bad: Committing to a machine instance for 1 to 3 years on the cloud might sound like an oxymoron. Organizations are moving to the cloud to get computing resources on demand. A long-term financial commitment flies in the face of this flexibility. Many AWS users take on-demand prices as a given, and pay the price of flexibility. Principle #4: Free Usage Tier The Principle: Amazon grants 1 year of free usage with generous quotas for many of its services, to reduce risk and encourage cloud adoption. This was a primary way AWS gained its initial market share in the early years. The Free Tier Grants (as of the time of this writing) 1 year of usage with 750 hours of EC2 instances, 750 hours of RDS usage (can run managed databases like MySQL), 5 GB on S3, 1 million requests on the cool serverless delivery platform Lambda, 50 GB storage on CloudFront (delivery network), 5GB on EFS (file storage), 30GB on EBS (block storage), 750 hours of ElasticSearch, and more. The Bad: The free tier has helped many organizations and technologists get “hooked” on Amazon’s offerings - it is a showcase of the astounding depth, breadth and technical excellence of their service profile. Amazon provides - and encourages - an enormous amount of sophistication within its ecosystem. It provides power, but power brings with it responsibility, overhead and a high cost of skills. Very often, organizations select AWS by default because it is a market leader and the option most well known by their teams. I can’t say the free tier is bad, but it has creatd an unfair advantage vs. other cloud offerings, which have their own strengths. Wrap Up AWS is great, but it is also a business and has established pricing that safeguards its interests. Carefully consider the benefits and tradeoffs of the Amazon pricing philosophy before entering a large-scale engagement. If you're already heavily engaged, plan your cloud consumption 1-2 years ahead and see if other cloud platforms - such as Azure or Google Cloud Platform - can give Amazon a run for its money.
This is an objective summary of the features and drawbacks of Hadoop/HDFS as an analytics platform and compare these to the cloud-based Snowflake data warehouse.
CORE
