Support Questions
Find answers, ask questions, and share your expertise

Hive LLAP queries taking down HDP cluster


We have an HDP cluster (version 2.6.5) and we primarily use it for hive LLAP queries. We have experienced an issue where executing certain queries takes down the entire cluster.

Here is the sequence of events.

  • A complex query is submitted - Usually involves multi-level sub-queries (4 levels) along with aggregations.
  • More than one instance of a query with the above conditions running concurrently.
  • These conditions trigger an insane amount of logging - Looks like hive llap query logging.
  • This leads to generating 100GB per minute of disk space and the CPU spikes on many slave nodes.
  • At this point, the only way to recover from this is to restart hive(llap) service.