Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
avatar
Cloudera Employee
  1. As the extras features are not shipped with the default log4j implementation hence the users will need to download the "Apache Extras™ for Apache log4j" (Like: apache-log4j-extras) : https://logging.apache.org/log4j/extras/download.html

    Example: For example download the jar "apache-log4j-extras-1.2.17.jar" and place it inside the location:

    /etc/hadoop/conf/secure/

    2. Add the property -Dhadoop.root.logger=INFO,ZIPRFA to export AMBARI_JVM_ARGS parameter in /var/lib/ambari-server/ambari-env.sh:

    export AMBARI_JVM_ARGS=$AMBARI_JVM_ARGS' -Xms512m -Xmx2048m -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Dhadoop.root.logger=INFO,ZIPRFA'

    3. Uncomment/Add the server classpath in /var/lib/ambari-server/ambari-env.sh:

    export SERVER_CLASSPATH=/etc/hadoop/conf/secure

    4. Added the following property to log4j:

    #log4j.rootLogger=INFO,filelog4j.rootLogger=INFO,ZIPRFA

    Comment out the following values:

    # Direct log messages to a log file
    #log4j.appender.file=org.apache.log4j.RollingFileAppender
    #log4j.appender.file.File=${ambari.log.dir}/${ambari.log.file}
    #log4j.appender.file.MaxFileSize=80MB
    #log4j.appender.file.MaxBackupIndex=60
    #log4j.appender.file.layout=org.apache.log4j.PatternLayout
    #log4j.appender.file.layout.ConversionPattern=%d{DATE} %5p [%t] %c{1}:%L - %m%n

    Add the following values:

    #### New Appender to Zip the Log Files Based on Daily Rotation
    log4j.appender.ZIPRFA=org.apache.log4j.rolling.RollingFileAppender
    log4j.appender.ZIPRFA.File=${ambari.log.dir}/${ambari.log.file}
    log4j.appender.ZIPRFA.layout=org.apache.log4j.PatternLayout
    log4j.appender.ZIPRFA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%
    log4j.appender.ZIPRFA.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
    log4j.appender.ZIPRFA.rollingPolicy.ActiveFileName=${ambari.log.dir}/${ambari.log.file}
    log4j.appender.ZIPRFA.rollingPolicy.FileNamePattern=${ambari.log.dir}/${ambari.log.file}-.%d{yyyyMMdd}.log.gz
    

    5. Run ps -ef | grep ambari and check for -Dhadoop.root.logger=INFO,EWMA,RFA

    6. Now restart ambari server and check for -Dhadoop.root.logger again, it should have been changed to -Dhadoop.root.logger=INFO,ZIPRFA

    Now when the date changes, ambari server logs would be rolled over.

    -rw-r--r--.  1 root root      301 Sep  8 12:33 ambari-server.log-.20170907.log.gz
    -rw-r--r--.  1 root root      921 Sep  9 20:23 ambari-server.log-.20170908.log.gz
    -rw-r--r--.  1 root root      249 Sep 10 03:05 ambari-server.log-.20170909.log.gz
    -rw-r--r--.  1 root root      304 Sep 11 05:43 ambari-server.log-.20170910.log.gz
    -rw-r--r--.  1 root root      247 Sep 12 17:55 ambari-server.log-.20170911.log.gz
    -rw-r--r--.  1 root root    28867 Sep 13 01:42 ambari-server.log-.20170912.log.gz
    -rw-r--r--.  1 root root     1608 Sep 14 00:35 ambari-server.log-.20170913.log.gz
    -rw-r--r--.  1 root root     1873 Sep 15 00:08 ambari-server.log-.20170914.log.gz
    Reference Article: https://community.hortonworks.com/articles/50058/using-log4j-extras-how-to-rotate-as-well-as-zip-th....
2,048 Views