Recreating an HBase Table Without Violating Region Starting Keys
Join the DZone community and get the full member experience.
Join For FreeRecently, I was using completebulkload to load a large amount of data into HBase. To keep this process well-balanced over the entire cluster (and thus faster), I was loading data into a table with pre-created regions. Since my bulkloading process failed in the middle a couple of times (due to some misconfiguration), I needed to truncate the table over and over.
I have noticed that a command like
hbase(main):017:0> truncate 'table'
will disable, drop and recreate the table with the same name settings (number of column familiers, compression, ttl, blocksize etc), but it does not maintain the region boundaries. It means that if you have a table with some number of regions and then truncate it, the table will be recreated with a single one region e.g.
hbase(main):018:0> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']} hbase(main):019:0> truncate 't1' Truncating 't1' table (it may take a while): - Disabling table... - Dropping table... - Creating table...
If you look at http://${hbase-master}:60010/table.jsp?name=t1, you will find that truncating makes table t1 empty and reduces the number of regions to single one.
Actually, I needed a functionality to truncate a table, but not violate the region and their starting keys (since I have choosen them so carefully earlier to balance the load over all region servers). I've implemented simple script for this purpose:
include Java import org.apache.hadoop.hbase.client.HTable import org.apache.hadoop.hbase.util.Bytes import org.apache.hadoop.hbase.client.HBaseAdmin import org.apache.hadoop.conf.Configuration import org.apache.hadoop.hbase.HTableDescriptor table_name = ARGV[0] table = HTable.new(table_name) region_start_keys = table.getStartKeys() table.close() admin = HBaseAdmin.new(Configuration.new()) table_descriptor = admin.getTableDescriptor(Bytes.toBytes(table_name)) admin.disableTable(table_name) admin.deleteTable(table_name) admin.createTable(table_descriptor, region_start_keys)
You may use it in the following day:
$ hbase org.jruby.Main region-keys.rb t1
The table should be successfully recreated with the same region boundaries, so that it will look the same as before, but it will be empty.
Published at DZone with permission of Adam Kawa, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Trending
-
Using Render Log Streams to Log to Papertrail
-
Cucumber Selenium Tutorial: A Comprehensive Guide With Examples and Best Practices
-
A React Frontend With Go/Gin/Gorm Backend in One Project
-
Extending Java APIs: Add Missing Features Without the Hassle
Comments