06-01-2018 03:25 AM - last edited on 06-01-2018 05:59 AM by cjervis
I am facing a weired problem across all the env. today i tried to add one column to hive table with cascade command to reflect in all existing partitions. it took 6 hours to update 4500 partitions.
even if i do compute incremenal stats from impala for the first time it took 6 hours to do that.
the hive metastore is in oracle.
looks like the query is not getting executed at the speed and somewhere its going in wait at database oracle server.
or it may also possible that hive metastore is not sending all the queries to oracle .
what could be the reason, i am not sure whats going on, any help will be appreciated.
06-08-2018 04:04 AM
06-08-2018 04:24 AM
I have done all the analysys and found below things :
1. involved DBA to check if there is any problem at Oracle where my metastore is, but can see there is no problem
all of the queries have finished normally,did not see any thread contention or wait time also.
2. then i tried to put logger level to all in Hive log4j properties and i was able to see all the actual queries being run to hive meta store. and here i can see there is no problem becuase all the queries are being executed in ms of times.
so it looks that there is no problem at HMS Oracle.
also i tried doing compute incremental stats from Impala and same problem i can see.