Can we export hive table to s3 or nfs share

[#1]

Hello Team,

can we import/export hive table to s3 location or nfs share ? as per what i searched it is only supported for hdfs location. can you please help me to know if we can do it and if yes how can we achieve it ?

[#2]

@jayes 

Unfortunately the Hive Import/Export is only supported for HDFS.  The only method I know of to get the table and data into S3 is as follows, see example below.

You need to create a table that is mapped onto S3 bucket and directory

CREATE TABLE tests3 (
id BIGINT, time STRING, log STRING
)
row format delimited fields terminated by ','
lines terminated by '\n'
STORED AS TEXTFILE
LOCATION 's3n://bucket/directory/';
Insert data into s3 table and when the insert is complete the directory will have a csv file

INSERT OVERWRITE TABLE tests3
select id, time, log
from testcsvimport;

[#3]

Thanks for the help. 

Also have a doubt can we export the data to hdfs in compressed format i.e., into zip file or something else

[#4]

Please see the accepted compression formats supported below.

https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/managing-clusters/topics/cm-choosing-configur...

When exporting in hive it will compress the data.

[#5]

Hi @MGreen ,

i am unable to find how to pass which type of format we want the export to get dumped.

is it using beeline if yes can you please help me with the command or the argument to pass to export in compress format.

since today when i run export command the data size of table and data size of exported data approximately same.

[#6 · Accepted solution]

@jayes 

Unfortunately there is no compression setting for the Hive Export.  This feature was introduced when Hive CLI was used in the HDP days of Hortonworks.

You will need to create your tables with compression enabled, and in your case you will need to either do one of the following.

Alter the table and add compression to the table properties, and then do an insert overwrite to the table to compress it.

Or create a new table with compression added to the table properties, and then insert the data from the old table into the new one.

I would recommend using the Snappy Compression.

[#7]

Thanks @MGreen for the help