Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Inserting to text table compressed

SOLVED Go to solution

Inserting to text table compressed

Contributor

I see no way to specify a compression default in the create table statement, so I tried:

 

SET COMPRESSION_CODEC=gzip;insert overwrite <text_table> select .. from <another_table>

 

and got "Writing to compressed text table is not supported. Use query option ALLOW_UNSUPPORTED_FORMATS to override." but ALLOW_UNSUPPORTED_FORMATS shouldn't be used according to docs.

 

Is there a trick to having impala write text files compressed?

 

-Mauricio

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Inserting to text table compressed

Guru
Impala won't be able to create gzip compression format for text file. Please refer to below documentation:

https://www.cloudera.com/documentation/enterprise/latest/topics/impala_file_formats.html

It mentioned below:

For text format, if LZO compression is used, you must create the table and load data in Hive.

If other kinds of compression are used, you must load data through LOAD DATA, Hive, or manually in HDFS.

So the short answer is that you can't do it in Impala.
1 REPLY 1
Highlighted

Re: Inserting to text table compressed

Guru
Impala won't be able to create gzip compression format for text file. Please refer to below documentation:

https://www.cloudera.com/documentation/enterprise/latest/topics/impala_file_formats.html

It mentioned below:

For text format, if LZO compression is used, you must create the table and load data in Hive.

If other kinds of compression are used, you must load data through LOAD DATA, Hive, or manually in HDFS.

So the short answer is that you can't do it in Impala.