How Twitter Does MySQL - Get Their Fork

Twitter is a big proponent of knowledge sharing and open source software.  It's also well known that Twitter moved away from Casandra, and turned back towards relational databases, like MySQL.  But with the number of users and large amounts of data that Twitter handles, they have had to make some impressive additions and changes in creating their own "flavor" of MySQL database, based on MySQL 5.5.  Now you can see how they were able to take MySQL so far as they have announced that everyone can get a taste of what Twitter's been cooking, by releasing their MySQL fork on GitHub.  Here are some of the changes they've made:

• Add additional status variables, particularly from the internals of InnoDB. This allows us to monitor our systems more effectively and understand their behavior better when handling production workloads.
• Optimize memory allocation on large NUMA systems: Allocate InnoDB's buffer pool fully on startup, fail fast if memory is not available, ensure performance over time even when server is under memory pressure.
• Reduce unnecessary work through improved server-side statement timeout support. This allows the server to proactively cancel queries that run longer than a millisecond-granularity timeout.
• Export and restore InnoDB buffer pool in using a safe and lightweight method. This enables us to build tools to support rolling restarts of our services with minimal pain.
• Optimize MySQL for SSD-based machines, including page-flushing behavior and reduction in writes to disk to improve lifespan.

-- Twitter Engineering

Of course, since this software is open source, there are no guarantees of bug fixes or maintenance issues, and it is not intended for use outside of Twitter (it's really designed to let people take a look at what Twitter has done to MySQL).  However, having such a powerful persistent storage technology at your fingertips is a rare opportunity, so be sure to check it out.

Twitter also employs a MySQL sharding framework, known as Gizzard, that handles data replication, cluster expansion and data migration, and is also open source. 

You can learn more information and download Twitter's MySQL Fork at GitHub.