What are hive's operation logs used for?

[#1]

In hive, there is the option to enable or disable operation logging: https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-HiveServer2L...

What are these logs for?

Some background on why I'm asking:

We recently ran into an issue where hiveserver2 was crashing due to huge number of open operation log files (same issue as https://community.hortonworks.com/questions/48351/hiveserver2-hive-users-nofile-ulimit-above-64000.h...). The files all appear to be empty AND they are just being kept open indefinitely. We are going to disable the logs by setting hive.server2.logging.operation.enabled to false but want to know the impact of doing so.

[#2]

@Kit Menke From what I can surmise the Hiveserver2 logging only applies to beeline sessions. Also, when you enable logging you would also need to enable the logging level in order to see any log entries https://issues.apache.org/jira/browse/HIVE-10119. Problem is I can't find those settings in either HDP 2.4 or 2.5.

[#3]

I've found two places so far where the operation logs are being used:

• Ambari Hive View -> Logs tab normally displays output like the number of mappers/reducers.
  • After disabling operation logging the log tab displays nothing.
• Beeline connections -> After executing a query normally some output is displayed.
  • After disabling operation logging there is no output except the results of the query.

Would be nice to get these back without compromising hiveserver2!

[#4]

hello, anyone knows the root cause of why there are so many open operation log files ? Any way to fix that ?