Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

insert into S3 hive table very slow

Highlighted

insert into S3 hive table very slow

Explorer

I want to load data from a hive table linking to HDFS location into a hive table linking to S3 location. But it took really long time. Below is the SQL statement. Can anyone give some tips for optimising the performance? Thank you very much!

insert overwrite table inventory_10 partition(inv_date_sk) select * from tpcds_bin_partitioned_orc_10.inventory;

# inventory_10 links to S3 location

# tpcds_bin_partitioned_orc_10.inventory links to HDFS location

1 REPLY 1

Re: insert into S3 hive table very slow

Explorer

Are you using s3a or s3n? What HDP version is it? Let me know if this article helps: http://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.11.1/bk_hdcloud-aws/content/s3-hive...

Don't have an account?
Coming from Hortonworks? Activate your account here