DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • 7 Essential Tips for a Production ClickHouse Cluster
  • Recovering an MS SQL Database From Suspect Mode: Step-By-Step Guide
  • Simplify Database Geo-Redundancy Backup With Cloud Storage Services
  • How to Build Your Exchange Server Recovery Strategy to Overcome Ransomware Attacks

Trending

  • Operational Principles, Architecture, Benefits, and Limitations of Artificial Intelligence Large Language Models
  • The Smart Way to Talk to Your Database: Why Hybrid API + NL2SQL Wins
  • Breaking Bottlenecks: Applying the Theory of Constraints to Software Development
  • Event-Driven Architectures: Designing Scalable and Resilient Cloud Solutions
  1. DZone
  2. Data Engineering
  3. Big Data
  4. HBase Backup Process: Explained

HBase Backup Process: Explained

Learn about how to take HBase backup of HBase data and tables in Hortonworks Sandbox 2.5. and all of the different ways that a backup can be taken for HBase datasets.

By 
Jose Praveen user avatar
Jose Praveen
·
May. 02, 17 · Tutorial
Likes (10)
Comment
Save
Tweet
Share
12.3K Views

Join the DZone community and get the full member experience.

Join For Free

I am happy to post another blog post on how to take HBase backup of HBase data and tables in Hortonworks Sandbox 2.5.

Important: You may get the following error in the HBase shell.

ERROR: Can’t get master address from ZooKeeper; znode data == null

Just open Ambari (Hortonworks Sandbox)/Cloudera manager (Cloudera Sandbox) and start the Zookeeper service or HBase Master to get rid of the above error.

I will help you to understand the different ways that a backup can be taken for HBase datasets.

Export

HBase has a utility called Export that is used to export the data of the HBase table to plain sequence files in the HDFS folder. A MapReduce job gets created which will call the HBase cluster to get the data from the specified table and write it to the HDFS folder.

You can export data from HBase table to either the Local FIle System or HDFS.

Local File System just copies into my HDFS rather than my local file system. So, from HDFS, I need to move the part-m-00000 to local file system.

HDFS just copies to my HDFS.

[root@sandbox ~]# hbase org.apache.hadoop.hbase.mapreduce.Export "person" "/jthiyaga/"

...where "person" is the table name in HBase and "jose" is local file system directory.

2017-03-22 17:12:53,304 INFO [main] util.RegionSizeCalculator: Calculating region sizes for table "person".
2017-03-22 17:12:54,034 INFO [main] client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService
2017-03-22 17:12:54,034 INFO [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x15af6c30f1e0017
2017-03-22 17:12:54,041 INFO [main] zookeeper.ZooKeeper: Session: 0x15af6c30f1e0017 closed
2017-03-22 17:12:54,041 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2017-03-22 17:12:54,639 INFO [main] mapreduce.JobSubmitter: number of splits:1
2017-03-22 17:12:54,679 INFO [main] Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2017-03-22 17:12:55,331 INFO [main] mapreduce.JobSubmitter: Submitting tokens for job: job_1490198811520_0001
2017-03-22 17:12:59,111 INFO [main] impl.YarnClientImpl: Submitted application application_1490198811520_0001
2017-03-22 17:12:59,311 INFO [main] mapreduce.Job: The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1490198811520_0001/
2017-03-22 17:12:59,316 INFO [main] mapreduce.Job: Running job: job_1490198811520_0001
2017-03-22 17:14:17,273 INFO [main] mapreduce.Job: Job job_1490198811520_0001 running in uber mode : false

Import

You can also import the data from the local filesystem. A MapReduce job gets created that will call the import the data from the HDFS folder to the existing HBase table.

[root@sandbox ~]# hbase org.apache.hadoop.hbase.mapreduce.Import "person" "/jthiyaga/"
2017-03-22 17:27:56,304 INFO [main] impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2017-03-22 17:27:56,434 INFO [main] client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/172.17.0.2:8050
2017-03-22 17:27:56,776 INFO [main] client.AHSProxy: Connecting to Application History server at sandbox.hortonworks.com/172.17.0.2:10200
2017-03-22 17:28:06,379 INFO [main] input.FileInputFormat: Total input paths to process : 1
2017-03-22 17:28:06,605 INFO [main] mapreduce.JobSubmitter: number of splits:1
2017-03-22 17:28:07,507 INFO [main] mapreduce.JobSubmitter: Submitting tokens for job: job_1490198811520_0002
2017-03-22 17:28:09,156 INFO [main] impl.YarnClientImpl: Submitted application application_1490198811520_0002
2017-03-22 17:28:09,546 INFO [main] mapreduce.Job: The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1490198811520_0002/
2017-03-22 17:28:09,551 INFO [main] mapreduce.Job: Running job: job_1490198811520_0002
2017-03-22 17:28:47,656 INFO [main] mapreduce.Job: Job job_1490198811520_0002 running in uber mode : false
2017-03-22 17:28:47,878 INFO [main] mapreduce.Job: map 0% reduce 0%
2017-03-22 17:29:11,169 INFO [main] mapreduce.Job: map 100% reduce 0

CopyTable

The CopyTable will directly write the data into an existing HBase table. This can be done in the same cluster or a remote cluster.

Both clusters should be online in order to make this done.

[root@sandbox jthiyaga]# hbase org.apache.hadoop.hbase.mapreduce.CopyTable --new.name=personNew person

...where "person" is the existing table (with data) and "personNew" is a table that I created. The CopyTable will copy data from the "person" to the "personNew" table. Behind the scenes, a MapReduce job will run.

Snapshots

The above types need a fast reliable network connection to connect the remote cluster.

You can also use Snapshots to have less performance impact on your cluster.

Cluster Shutdown

This is a simple but not recommended type of HBase data backup, as you need to shut down the HBase cluster to stop data write to the cluster.

Just open hbase-site.xml. The following shows your table and data are being saved at hdfs dfs -ls /apps/hbase/data.

<property>
<name>hbase.rootdir</name>
<value>hdfs://sandbox.hortonworks.com:8020/apps/hbase/data</value>
</property>

The following shows the "personNew" table, which I imported using the Import option.

[root@sandbox conf]# hdfs dfs -ls /apps/hbase/data/data/default
Found 6 items
drwxr-xr-x - hbase hdfs 0 2016-09-13 10:59 /apps/hbase/data/data/default/ATLAS_ENTITY_AUDIT_EVENTS
drwxr-xr-x - hbase hdfs 0 2017-03-20 16:16 /apps/hbase/data/data/default/MyStream
drwxr-xr-x - hbase hdfs 0 2016-09-13 10:59 /apps/hbase/data/data/default/atlas_titan
drwxr-xr-x - hbase hdfs 0 2016-09-13 11:09 /apps/hbase/data/data/default/iemployee
drwxr-xr-x - hbase hdfs 0 2017-03-20 17:27 /apps/hbase/data/data/default/person
drwxr-xr-x - hbase hdfs 0 2017-03-22 17:38 /apps/hbase/data/data/default/personNew
[root@sandbox conf]# hdfs dfs -ls /apps/hbase/data/data/default/personNew
Found 3 items
drwxr-xr-x - hbase hdfs 0 2017-03-22 17:38 /apps/hbase/data/data/default/personNew/.tabledesc
drwxr-xr-x - hbase hdfs 0 2017-03-22 17:38 /apps/hbase/data/data/default/personNew/.tmp
drwxr-xr-x - hbase hdfs 0 2017-03-22 17:38 /apps/hbase/data/data/default/personNew/59acfc94838d84a3f1810efaacda0da8

Thanks for reading this post.

Database Backup File system cluster Data (computing) hadoop

Published at DZone with permission of Jose Praveen, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • 7 Essential Tips for a Production ClickHouse Cluster
  • Recovering an MS SQL Database From Suspect Mode: Step-By-Step Guide
  • Simplify Database Geo-Redundancy Backup With Cloud Storage Services
  • How to Build Your Exchange Server Recovery Strategy to Overcome Ransomware Attacks

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: