question Re: log4g + logs not rotated and zipped in Archives of Support Questions (Read Only)

log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 00:35:00 GMT

hi all

we configured the following properties in Advanced ams-hbase-log4j

log4j.appender.RFA.layout=org.apache.log4j.PatternLayout
log4j.appender.RFA=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.RFA.rollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy
log4j.appender.RFA.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.RFA.rollingPolicy.ActiveFileName=${hbase.log.dir}/${hbase.log.file}.log
log4j.appender.RFA.rollingPolicy.FileNamePattern=${hbase.log.dir}/${hbase.log.file}.%i.log.gz
log4j.appender.RFA.triggeringPolicy.MaxFileSize=1000000
log4j.appender.RFA.rollingPolicy.maxIndex=10

but after metrics collector restart

we not see that hbase-ams rotated and sipped to .gz

please advice what could be wrong with the log4j properties ?

example of the logs that we get ,

-rw-r--r--  1 ams hadoop     45434111 Aug 16 17:07 hbase-ams-master-master.isys.com.log
-rw-r--r--  1 ams hadoop     2802 Aug 16 17:07 hbase-ams-master-master.isys.com.out

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 00:56:59 GMT

@Michael Bronson

Hbase master does not come with apache-log4j-extras-1.2.17.jar jar file - hence it fails with error as "RollingFileAppender classnotfound error"

As a workaround I copied the JAR file hiveserver2 to hbase master and restart the service then it starts working.

cp /usr/hdp/current/hive-server2/lib/apache-log4j-extras-1.2.17.jar /usr/hdp/current/hbase-master/lib/

Please note that this is temporary workaround - If you will have to do this again in case of HDP upgrade.

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 01:02:01 GMT

@Michael Bronson

I am sorry - that workaround is for HBase-master - let me share the step for AMS Hbase master

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 01:05:09 GMT

@Michael Bronson

You can try with below step and restart the service and check

cp /usr/hdp/current/hive-server2/lib/apache-log4j-extras-1.2.17.jar /usr/lib/ambari-metrics-collector/

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 01:07:16 GMT

under /usr/hdp/current , we have only the follwing links ( not hbase-master folder )

hadoop-client            hadoop-hdfs-portmap             hadoop-yarn-client           hive-server2        livy-client           spark2-thriftserver  zookeeper-server
hadoop-hdfs-client       hadoop-hdfs-secondarynamenode   hadoop-yarn-nodemanager      hive-server2-hive2  pig-client            spark-client
hadoop-hdfs-datanode     hadoop-hdfs-zkfc                hadoop-yarn-resourcemanager  hive-webhcat        shc                   spark_llap
hadoop-hdfs-journalnode  hadoop-httpfs                   hadoop-yarn-timelineserver   kafka-broker        slider-client         storm-slider-client
hadoop-hdfs-namenode     hadoop-mapreduce-client         hive-client                  livy2-client        spark2-client         tez-client
hadoop-hdfs-nfs3         hadoop-mapreduce-historyserver  hive-metastore               livy2-server        spark2-historyserver  zookeeper-client

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 01:12:06 GMT

@Michael Bronson yes, you have hive-server2 - can copy from that to ams folder

cp /usr/hdp/current/hive-server2/lib/apache-log4j-extras-1.2.17.jar /usr/lib/ams-hbase/lib/

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 01:18:41 GMT

I copy the file and restart the service but without good results

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 01:32:23 GMT

its works -:)

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 01:33:36 GMT

another one question please , I see that ambari-metrics-collector.log created with huge size , can we rote it also and zipped it ?

-rw-r--r--  1 ams hadoop     2008 Aug 16 18:32 ambari-metrics-collector.out
-rw-r--r--. 1 ams hadoop 40776116 Aug 16 18:33 ambari-metrics-collector.log

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 01:39:44 GMT

@Michael Bronson

Thanks for confirming that it worked - you can update the log4j configurations and copy the jar file and then check it?

cp /usr/hdp/current/hive-server2/lib/apache-log4j-extras-1.2.17.jar /usr/lib/ambari-metrics-collector/

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 01:52:52 GMT

ams-log4j.txtafter I set the following and copy the files , and restart the service , still logs not rotated and ziped

can you please check if I wrong here ?

log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.file.rollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy
log4j.appender.file.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.file.rollingPolicy.ActiveFileName=${ams.log.dir}/${ams.log.file}.log
log4j.appender.file.rollingPolicy.FileNamePattern=${ams.log.dir}/${ams.log.file}.%i.log.gz
log4j.appender.file.triggeringPolicy.MaxFileSize=1000
log4j.appender.file.rollingPolicy.maxIndex=10

<br>

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 02:06:34 GMT

@Michael Bronson any WARNINGS in ambari-metrics-collector.out ?

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 02:11:00 GMT

more ambari-metrics-collector.out
log4j:WARN Failed to set property [triggeringPolicy] to value "org.apache.log4j.rolling.SizeBasedTriggeringPolicy".
log4j:WARN Failed to set property [rollingPolicy] to value "org.apache.log4j.rolling.FixedWindowRollingPolicy".
log4j:WARN Please set a rolling policy for the RollingFileAppender named 'file'
log4j:ERROR No output stream or file set for the appender named [file].
Aug 16, 2018 7:04:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider as a provider class
Aug 16, 2018 7:04:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices as a root resource class
Aug 16, 2018 7:04:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TimelineWebServices as a root resource class
Aug 16, 2018 7:04:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Aug 16, 2018 7:04:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.11 12/09/2011 10:27 AM'
Aug 16, 2018 7:04:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Aug 16, 2018 7:04:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton"
Aug 16, 2018 7:04:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton"
Aug 16, 2018 7:04:29 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton"

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 02:56:34 GMT

seems like last 3 lines are messed up. can you try like below?

log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.file.rollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy
log4j.appender.file.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.file.rollingPolicy.ActiveFileName=${ams.log.dir}/${ams.log.file}.log
log4j.appender.file.rollingPolicy.FileNamePattern=${ams.log.dir}/${ams.log.file}.%i.log.gz
log4j.appender.file.triggeringPolicy.MaxFileSize=1000
log4j.appender.file.rollingPolicy.maxIndex=10

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 03:09:54 GMT

yes I already set this lines see the previos attcment file

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 03:21:25 GMT

what is mean - log4j:ERROR No output stream or file set for the appender named [file]. ?

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 03:25:35 GMT

also


log4j:WARN Failed to set property [triggeringPolicy] to value "org.apache.log4j.rolling.SizeBasedTriggeringPolicy".
log4j:WARN Failed to set property [rollingPolicy] to value "org.apache.log4j.rolling.FixedWindowRollingPolicy".
log4j:WARN Please set a rolling policy for the RollingFileAppender named 'file'

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 03:51:14 GMT

Can you make sure this file is present?

/usr/lib/ambari-metrics-collector/apache-log4j-extras-1.2.17.jar

Re: log4g + logs not rotated and zipped

mike_bronson7 — Fri, 17 Aug 2018 04:05:00 GMT

ls -ltr /usr/lib/ambari-metrics-collector/apache-log4j-extras-1.2.17.jar
-rw-r--r-- 1 root root 448794 Aug 16 18:51 /usr/lib/ambari-metrics-collector/apache-log4j-extras-1.2.17.jar

Re: log4g + logs not rotated and zipped

apappu — Fri, 17 Aug 2018 07:31:16 GMT

@Michael Bronson am also seeing same issue in my setup- will try and let you know.