Transparent Sharding

If you’ve looked into the material published on our site you probably understand that the ScaleBase Database Load Balancer shards for write scalability.

Taken from Wikipedia(http://en.wikipedia.org/wiki/Sharding):

Database sharding is a method of horizontal partitioning in a database or search engine

Now, there are many limitations when it comes to sharding. Think about it for a second. How do you run a join operation? How to you maintain data integrity (via database foreign keys)? How can you know the number of rows in a sharded table?

These limitations have limited the popularity of sharding solutions. Also, the price associated with sharding has been high since the sharding logic has to be written by developers and requires application changes. If the application is using O/R mapping tools, this becomes even more difficult. The operations team now needs to support more machines. The data intelligence guys can’t run their reporting systems since they are unaware of the shards.

Curt Monash (from DBMS2 – I strongly suggest reading his blog) came up with the term Transparent Sharding (see http://www.dbms2.com/2011/02/24/transparent-sharding/) which I think fits ScaleBase perfectly. Basically, all the complexities associated with sharding are handled by ScaleBase Database Load Balancer and the developers are free to focus on their apps. Now, since what ScaleBase does is sharding, we took the time to solve many of the sharding related issues. Joins, for example, are handled by our use of master, global, and split tables (you can read more about it here – http://www.scalebase.com/split-master-and-global-tables/). Since we’re implementing the database network protocol, reporting tools can run against the ScaleBase Database Load Balancer and sys-ops can run their usual tuning and configuration scripts against the load balancer with no changes whatsoever.

So, if you’ve heard of Sharding, and want to implement sharding – try us out. I believe you’ll see the value of our off-the-shelf, transparent solution as a good alternative to an in-house development process.