After almost a week, we decided to downgrade hive version from HDP-2.5.0.0-tag to HDP-2.4.4.0-tag. As I can explain this is because of commit: https://github.com/hortonworks/hive-release/commit/059f15fe55977246da1e432cf18202b48789775d#diff-e5e61d7b4ede0525d16c819ae6d8f5c5 This commit was produced to improve speed on add partition operation. But it leads to thread-unsafe side-effects on the high-loaded cluster with many add partition operations. Unfortunately I can't find what exactly incorrect in this code or maybe it needs especial set of configuration parameters. Anyway to downgrade hive version we simply redirected all symbol links for hive* in /usr/hdp to hive HDP-2.4.4.0-tag directories, replaced hive jars in /user/oozie/share and restart all hive and oozie services. After that all services work normally.
... View more
