Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!
If you have started using Hive LLAP, you would have noticed that by default its configured to use log4j2.
Default configuration makes use of advanced features from log4j2 like Rolling Over logs based on time interval and size.
With time, a lot of old log files would have accumulated and typically you would compress those files manually or add additional jars and change configuration when using log4j1 to achieve the same
With log4j2, a simple change in configuration can ensure that every time a log file is rolled over, it gets compressed for optimal use of storage space.
Default configuration:
To automatically compress the rolled over log files, update the highlighted line to:
appender.DRFA.filePattern = ${sys:hive.log.dir}/${sys:hive.log.file}.%d{yyyy-MM-dd}-%i.gz-%i will ensure that in a rare scenario when there has been increased logging and the threshold size can be been reached more than once in the specified interval, the previously rolled over file won't get over written.
.gz will ensure that files are compressed using gzip
To understand the finer details about log4j2 appenders, you may check out the official documentation.
Similarly you can also make similar changes to llap-cli log settings:
