Over a million developers have joined DZone.

Bash Magic: List Hive Table Sizes in GB

How to list the sizes of Hive tables in Hadoop in GBs.

· Big Data Zone

Learn about how to rapidly iterate data applications, while reusing existing code and leveraging open source technologies, brought to you in partnership with Exaptive.

To list the sizes of Hive tables in Hadoop in GBs:


sudo -u hdfs hadoop fs -du /user/hive/warehouse/ | awk '/^[0-9]+/ { print int($1/(1024**3)) " [GB]\t" $2 }'



Result:

448 [GB] hdfs://aewb-analytics-staging-name.example.com:8020/user/hive/warehouse/mybigtable
8 [GB]hdfs://aewb-analytics-staging-name.example.com:8020/user/hive/warehouse/anotherone
0 [GB]hdfs://aewb-analytics-staging-name.example.com:8020/user/hive/warehouse/tinyone


The Big Data Zone is brought to you in partnership with Exaptive.  Learn how Rapid Application Development powers business. 

Topics:
architects,bigdata,bash,hive,big data

Published at DZone with permission of Jakub Holý , DZone MVB .

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}