Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there a way to define the HDFS maximum file size per table using hive (no. of files will be derived automatically from the definition)?

Highlighted

Is there a way to define the HDFS maximum file size per table using hive (no. of files will be derived automatically from the definition)?

New Contributor
 
1 REPLY 1

Re: Is there a way to define the HDFS maximum file size per table using hive (no. of files will be derived automatically from the definition)?

Super Guru

@Abraham Abraham

Do you mean the maximum size of each file while the table may have multiple files or maximum size of the table?

For maximum file size, you cannot do much except for a block size (each file having multiple blocks). There is no limit to a file size.

However, you can limit the size of data in each directory using HDFS disk quota. So assume you have external table at /user/mytable

Now you can the quota for this directory to be 1TB for example. In this case you will limit the table size to be 1TB. But you can of course have multiple files.

Don't have an account?
Coming from Hortonworks? Activate your account here