Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Highlighted

Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Explorer

Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

7 REPLIES 7
Highlighted

Re: Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Rising Star

The individual file sizes is not stored in the metastore, so there is not a way to query them directly

From within the hive shell you can execute HDFS commands such as

dfs -ls /path/to/table 

to see the individual files and their sizes.

If you're interested in the total data size of the table, you can execute:

 DESCRIBE FORMATTED table_name;  

and look for the table parameter named totalSize.

Highlighted

Re: Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

New Contributor

I want to see the size of all the table in hive residing in multiple databases in. There are around 3000 tables, so it is difficult to do it one by one for each table. How can I do it at one go?

 

Regards,

Manu

Re: Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Community Manager

@ManuN 

As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. 


Vidya Sargur, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Highlighted

Re: Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Expert Contributor

Access the path where these files are stored to find the size of these split files. You can do a DESCRIBE EXTENDED /FORMATTED Tablename to find the exact path of the files.

Highlighted

Re: Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Explorer

@Benjamin Hopp DESCRIBE FORMATTED table_name; not giving the individual file size.

Highlighted

Re: Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Expert Contributor

@Stinger

It won't give you the file size, but it will give you the path where those files are stored, so you can refer that path for details like file size, permissions, etc.

Highlighted

Re: Hive tables are split into files, how can we know the size of the each file by using hive shell( query)

Expert Contributor

get hdfs path where hive table files are stored. Use hdfs dfs -du -s -h /hdfs_path to get size in readable format.

Don't have an account?
Coming from Hortonworks? Activate your account here