Member since
05-20-2017
12
Posts
1
Kudos Received
0
Solutions
05-15-2018
03:59 PM
1 Kudo
Unfortunately "--hive-overwrite" option destroy hive table structure and re-create it after that which is not acceptable way. The only way is: 1. hive> truncate table sample; 2. sqoop import --connect jdbc:mysql://yourhost/test --username test --password test01 --table sample --hcatalog-table sample
... View more
10-11-2017
12:38 PM
@Aditya Sirna That's it. Thank you so much.
... View more
10-10-2017
03:34 PM
Finally, I found the solution.
set hive.auto.convert.join.noconditionaltask = true;
set hive.auto.convert.join.noconditionaltask.size = 2000000; By playing with hive.auto.convert.join.noconditionaltask.size got adequate performance. Low value provides performance degradation. Next parameters also might be helpful: set hive.auto.convert.sortmerge.join=true
set hive.optimize.bucketmapjoin=true
set hive.optimize.bucketmapjoin.sortedmerge=true
set hive.auto.convert.sortmerge.join.noconditionaltask=true
set hive.auto.convert.sortmerge.join.bigtable.selection.policy=org.apache.hadoop.hive.ql.optimizer.TableSizeBasedBigTableSelectorForAutoSMJ
... View more
05-20-2017
09:04 PM
Understood. Thank you.
... View more