Reply
Explorer
Posts: 46
Registered: ‎09-29-2016

compute stats or alter statement taking long time

[ Edited ]

when doing compute incremental stats on a table having 4000 partitions in total and 30 gb of data takes 5-6 hrs to complete.

have seen hive.log which is executing lots of queries to update metastore . 

i have cloudera 5.7 with oracle as metastore.

 

have seen same thing gets completed in 2-3 min in MySQL metastore.

but cant say oracle is creating problem becuase have seen with DBA that everything is smooth at Oracle.

looks some problem at impala/hive.

Explorer
Posts: 46
Registered: ‎09-29-2016

Re: compute stats or alter statement taking long time

new updates,

I created separate cluster where there is only impala and hive then its working fine,compute stats is getting comleted in 10 mins. but when hadoop,impala and hive are together on same cluster its taking more than 1.5 hr to complete the compuet stats.

 

Now this is really surprising when hadoop,imapala and hive are on same cluster it should be actually faster as data will be local to hive/impala.

 

people any thought on this ?

Announcements