Support Questions
Find answers, ask questions, and share your expertise

insert into S3 hive table very slow

Explorer

I want to load data from a hive table linking to HDFS location into a hive table linking to S3 location. But it took really long time. Below is the SQL statement. Can anyone give some tips for optimising the performance? Thank you very much!

insert overwrite table inventory_10 partition(inv_date_sk) select * from tpcds_bin_partitioned_orc_10.inventory;

# inventory_10 links to S3 location

# tpcds_bin_partitioned_orc_10.inventory links to HDFS location

1 REPLY 1

Explorer

Are you using s3a or s3n? What HDP version is it? Let me know if this article helps: http://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.11.1/bk_hdcloud-aws/content/s3-hive...