Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Solved Go to solution

how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Can anyone let me know how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

I understand that Sqoop can export data to tables, however I have a scenario where I want to export data into flat files

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Thanks Artem and Neeraj. I just copied the data from /apps/hive/warehouse to a local directory.

Note: Hive data is not necessary located in the folder mentioned above. i.e. external table can be defined anywhere within the HDFS. hive –e “DESCRIBE FORMATTED <tablename>” > <tablename_descformat>.txt;

This command is to collect detail information of tables. The detail information for a table includes column, partition, database it belongs, owner, create time, last access time, protect mode, location, table type, storage information and etc.

View solution in original post

8 REPLIES 8
Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Mentor

unless you're using orc, the files you use for Hive are in their raw form. You can just browse to /apps/hive/warehouse dir and look at them. No need to use sqoop.

Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Thanks for reply. I'm using orc format

Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Mentor

@vijaya inturi you can also read orc and output to flat file using pig

A = load ‘student.orc’ using OrcStorage();

Store A into 'file' using PigStorage([options]);

Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

@vijaya inturi

You can use this to write into HDFS from existing table

INSERT OVERWRITE DIRECTORY '/path/to/output/dir' SELECT * FROM table

Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Thanks Neeraj. Is there any command to export all the data from the hive database into a flat file?

Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Thanks Artem and Neeraj. I just copied the data from /apps/hive/warehouse to a local directory.

Note: Hive data is not necessary located in the folder mentioned above. i.e. external table can be defined anywhere within the HDFS. hive –e “DESCRIBE FORMATTED <tablename>” > <tablename_descformat>.txt;

This command is to collect detail information of tables. The detail information for a table includes column, partition, database it belongs, owner, create time, last access time, protect mode, location, table type, storage information and etc.

View solution in original post

Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Mentor

yes correct, but if table is already external, then I don't see a point in need to sqoop data out of Hive as you can just work with those files directly.

Highlighted

Re: how to use Sqoop to export data from Hive to a flat file to be picked up by Hive again?

Mentor

also as a good practice, since you achieved the intended result, please accept one of the answers you see fit to close out the thread.

Don't have an account?
Coming from Hortonworks? Activate your account here