06-08-2018 04:30 AM - last edited on 06-08-2018 06:16 AM by cjervis
when doing compute incremental stats on a table having 4000 partitions in total and 30 gb of data takes 5-6 hrs to complete.
have seen hive.log which is executing lots of queries to update metastore .
i have cloudera 5.7 with oracle as metastore.
have seen same thing gets completed in 2-3 min in MySQL metastore.
but cant say oracle is creating problem becuase have seen with DBA that everything is smooth at Oracle.
looks some problem at impala/hive.
08-07-2018 10:58 PM
I created separate cluster where there is only impala and hive then its working fine,compute stats is getting comleted in 10 mins. but when hadoop,impala and hive are together on same cluster its taking more than 1.5 hr to complete the compuet stats.
Now this is really surprising when hadoop,imapala and hive are on same cluster it should be actually faster as data will be local to hive/impala.
people any thought on this ?