Benchmarking Aurora vs MySQL: Is Amazon’s DB Really 5x Faster?

in this post we’re taking a closer look at amazon aurora to see how it fares against mysql and if it delivers on the promise of a 5x performance boost. recently reaching general availability , aurora is amazon’s home grown mysql compatible database. currently available on three aws regions (virginia, oregon and ireland) on top of rds, amazon’s relational database service.

java / scala developer? takipi replaces logging in production jvms and lets you see the variable state that caused each log error and exception.

aurora is basically the next stage in the evolution of amazon’s hosted services. if now we’re used to using ec2 for our servers, and rds was introduced for optimized database instances, aurora is the next step in this offering. fully compatible with any operation you’d run through mysql so no code changes are needed.

new post: benchmarking aurora vs mysql: is amazon’s new db really 5x faster? http://t.co/0acjmwkdde pic.twitter.com/unq0m7zoll

— takipi (@takipid) september 17, 2015

to see a previous speed test we ran and discover how each of the regions compare to each other, you can check out this blog post right here .

why amazon is building a new mysql compliant database?

unlike hosting a database on rds, where we have additional performance benefits over ec2, monitoring, and quick installation features, aurora even has some newer tricks up its sleeve. seems like a new database was required to make it easier for amazon to further optimize performance and add features like backups and autoscaling database storage – which is not available on the basic mysql instances. in that sense, we’re promised that aurora would act like mysql if it was optimized to run on aws from day 1.

a side effect of this makes aurora available only on aws. so for example, you’d not be able to use a local version for testing and would have to completely rely on aws instances. on the bright side, migrating to aurora from an existing database instance is just a matter of a few clicks.

benchmark results

to have more of a hands on feel of how aurora stands up to mysql performance-wise we’ve set up a simple benchmark covering a write-heavy scenario using sysbench . to do this, we’ve used a similar configuration from a previous aurora performance assessment ran by amazon. in this version of the benchmark, we ran 4 different instances: the latest mysql 5.6 on ec2 and on rds, aurora on rds, and the release candidate version of the upcoming mysql 5.7 on ec2. the exact setup we used and the raw results are available on the next section of this post. let’s jump straight ahead and have a look at the findings:

operations per second

comparing operations per second for mysql and aurora on ec2 and rds

1. mysql 5.6 was 8% faster than aurora – the new database didn’t provide the best results on this test, and far off from the 5x promise we’re used to hear from amazon. perhaps the biggest differences kick in on different setups with bigger db sizes, but this just comes to show us the importance of trying it out with your specific use case.

2. rds provides up to a 60% performance benefit over ec2 – a more significant difference is seen when we compare performance of databases on ec2 vs rds. using rds provided considerable improvement over deploying the same database on ec2.

3. mysql 5.7 rc provided slightly better results than mysql 5.6 – the next version of mysql shows some progress, even though the rc benchmark run contained a considerable amount of ignored errors (full results are available through the link below).

4. when taking 95th percentile response times into account, aurora takes the lead – aurora was 20% better than mysql 5.6 on rds, and mysql 5.7 was 20% better than mysql 5.6 on ec2. rds performed about 3x better than ec2.

** [edit] please note that optimizing mysql’s my.cnf configuration would have probably improved its results. thanks @morgo !

response times

comparing 95h percentile response times for mysql and aurora on ec2 and rds

the groundwork

the benchmark included setting up 5 r3.large instances on aws: an ec2 machine which will run the benchmark, and machines with the databases themselves. mysql 5.6 on ec2, mysql 5.7 rc on ec2, mysql 5.6 on rds and aurora which is only available on rds. then, we’ve created a test database with 250 tables and a 25,000 table size on each of the tested machines:

./sysbench –test=tests/db/oltp.lua –mysql-host= –mysql-port=3306 –mysql-user= –mysqlpassword= –mysql-db= –mysql-table-engine=innodb — oltp-table-size=25000 –oltp-tables-count=250 –db-driver=mysql prepare

afterwards we’ve ran a variation of amazon’s write-heavy benchmark described here (300 sec instead of 600 and 125 threads instead of 1000):

./sysbench –test=tests/db/oltp.lua –mysql-host= –oltp-tables-count=250 –mysql-user= –mysqlpassword= –mysql-port=3306 –db-driver=mysql –oltp-tablesize=25000 –mysql-db= –max-requests=0 –max-time=300 — oltp_simple_ranges=0 –oltp-distinct-ranges=0 –oltp-sum-ranges=0 –oltporder-ranges=0 –oltp-point-selects=0 –num-threads=125 –randtype=uniform run

so basically, the benchmark had 5 minutes to throw all it’s got at each of the database instances, and report back on the results. the raw sysbench output is available right here:

– mysql 5.6 on ec2
– mysql 5.7 rc on ec2
– mysql 5.6 on rds
– aurora on rds

final thoughts

as far as managed services go, aurora is definitely a step in the right direction. taking on hosted solutions for databases to the next level. but was it really necessary to create a new database to support autoscaling? we feel that rds proved that it’s the right way to go as far as hosted mysql goes and it’s a good time to experiment with aurora if you’re containing it to specific cases for testing in production. like with any new technology that gets so close to the data at the core of your business, you’ll need to proceed with caution. with that said, a main pain point we didn’t address here is pricing and this is where aurora holds the greatest promise against traditional full blown commercial databases.

java / scala developer? takipi replaces logging in production jvms and lets you see the variable state that caused each log error and exception.