Community Articles
Find and share helpful community-sourced technical articles
Labels (1)
Cloudera Employee
  1. As the extras features are not shipped with the default log4j implementation hence the users will need to download the "Apache Extras™ for Apache log4j" (Like: apache-log4j-extras) : https://logging.apache.org/log4j/extras/download.html

    Example: For example download the jar "apache-log4j-extras-1.2.17.jar" and place it inside the location:

    /etc/hadoop/conf/secure/

    2. Add the property -Dhadoop.root.logger=INFO,ZIPRFA to export AMBARI_JVM_ARGS parameter in /var/lib/ambari-server/ambari-env.sh:

    export AMBARI_JVM_ARGS=$AMBARI_JVM_ARGS' -Xms512m -Xmx2048m -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Dhadoop.root.logger=INFO,ZIPRFA'

    3. Uncomment/Add the server classpath in /var/lib/ambari-server/ambari-env.sh:

    export SERVER_CLASSPATH=/etc/hadoop/conf/secure

    4. Added the following property to log4j:

    #log4j.rootLogger=INFO,filelog4j.rootLogger=INFO,ZIPRFA

    Comment out the following values:

    # Direct log messages to a log file
    #log4j.appender.file=org.apache.log4j.RollingFileAppender
    #log4j.appender.file.File=${ambari.log.dir}/${ambari.log.file}
    #log4j.appender.file.MaxFileSize=80MB
    #log4j.appender.file.MaxBackupIndex=60
    #log4j.appender.file.layout=org.apache.log4j.PatternLayout
    #log4j.appender.file.layout.ConversionPattern=%d{DATE} %5p [%t] %c{1}:%L - %m%n

    Add the following values:

    #### New Appender to Zip the Log Files Based on Daily Rotation
    log4j.appender.ZIPRFA=org.apache.log4j.rolling.RollingFileAppender
    log4j.appender.ZIPRFA.File=${ambari.log.dir}/${ambari.log.file}
    log4j.appender.ZIPRFA.layout=org.apache.log4j.PatternLayout
    log4j.appender.ZIPRFA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%
    log4j.appender.ZIPRFA.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
    log4j.appender.ZIPRFA.rollingPolicy.ActiveFileName=${ambari.log.dir}/${ambari.log.file}
    log4j.appender.ZIPRFA.rollingPolicy.FileNamePattern=${ambari.log.dir}/${ambari.log.file}-.%d{yyyyMMdd}.log.gz
    

    5. Run ps -ef | grep ambari and check for -Dhadoop.root.logger=INFO,EWMA,RFA

    6. Now restart ambari server and check for -Dhadoop.root.logger again, it should have been changed to -Dhadoop.root.logger=INFO,ZIPRFA

    Now when the date changes, ambari server logs would be rolled over.

    -rw-r--r--.  1 root root      301 Sep  8 12:33 ambari-server.log-.20170907.log.gz
    -rw-r--r--.  1 root root      921 Sep  9 20:23 ambari-server.log-.20170908.log.gz
    -rw-r--r--.  1 root root      249 Sep 10 03:05 ambari-server.log-.20170909.log.gz
    -rw-r--r--.  1 root root      304 Sep 11 05:43 ambari-server.log-.20170910.log.gz
    -rw-r--r--.  1 root root      247 Sep 12 17:55 ambari-server.log-.20170911.log.gz
    -rw-r--r--.  1 root root    28867 Sep 13 01:42 ambari-server.log-.20170912.log.gz
    -rw-r--r--.  1 root root     1608 Sep 14 00:35 ambari-server.log-.20170913.log.gz
    -rw-r--r--.  1 root root     1873 Sep 15 00:08 ambari-server.log-.20170914.log.gz
    Reference Article: https://community.hortonworks.com/articles/50058/using-log4j-extras-how-to-rotate-as-well-as-zip-th....
1,529 Views