Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Orc table with text file

Hive Orc table with text file

Explorer

Hi,

If hive table is created in ORC format, should the source file to be loaded into Hive ORC table has to be in ORC format ?

load data inpath '/path/file.txt' into table hiveorctable;

This is not working for me. Error: Invalid postscript.

Thanks

6 REPLIES 6

Re: Hive Orc table with text file

Contributor

If you have text data, create source table (hive_text_table) in textfile format. Load data using the command you specified. Create another orc table(e.g hive_orc_table) "stored as orc" and run "insert overwrite table hive_orc_table select * from hive_text_table"

Re: Hive Orc table with text file

New Contributor

Yes.It should be ORC file as specified in the creation of table definition.What we are specified file type at the time of creation should be load.

Other wise do one thing.Create a temp table with stored as text(by default if not specified) and later create your table with stored as orc using select from temp table(ctas)

Re: Hive Orc table with text file

Guru

@Ram You can not load a text file into an ORC table directly. The typical practice is to create a temporary Hive table, load the text file into the temporary table, then let Hive populate the ORC table:

1. create external table mytable_temp as (...schema...);
2. load data inpath '/path/to/file.txt' into table mytable_temp;
3. insert into table mytable_orc select * from mytable_temp;
4. drop table mytable_temp;

Re: Hive Orc table with text file

Expert Contributor

In your doc there is a good schema that summarises what you have just said Rich. Let's complement your answer with it:

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_dataintegration/content/moving_data_from_...

Highlighted

Re: Hive Orc table with text file

Explorer

Yes Thanks guys for the reply. I loaded Hive text table using text file and then tried to insert into Hive ORC table.

Just a single record into the table. It is running for long time with no output. Ran it in Hive CLI mode.

Total jobs = 3 Launching Job 1 out of 3 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_xxx,

Tracking URL = xxx

Kill Command = xxx

After this nothing is coming up. There is no response after the kill command.

Re: Hive Orc table with text file

Guru

@Ram - that sounds like a different question. Can you start a new question, and then maybe show us some of your code in your new post? You will get a better response in a new post. Thanks!