with the following API I restart all services in ambari that required restart curl -sH "X-Requested-By: ambari" -u USER:$PASSWD -i http://$SERVER:8080/api/v1/clusters/$CLUSTER_NAME/requests?fields=Requests/request_status how to know if API succeeded to restart all services ? for example we get true in case all services restart succsfuly we get false when one or more services are failed ... View more

we manage many hadoop clusters based on redhat OS ( version 7.x ) based on our experience ( many problem of low memory , disks performance , network problem , etc ) we agree that we need to install some monitoring tool that have ability to save the monitoring at least one month history details from the link below https://neverendingsecurity.wordpress.com/tag/atop/ we saw a lot of monitoring tool and we not sure what is the best tool for hadoop clusters , meanwhile we install the atop tool that its fine ( but take a lot of space under /var/log/atop ) **but we still thinking if this is good selecting** ... View more

hi all, we configured the Audit logging for ResourceManager , ( in YARN --> Advanced yarn-log4j ) as the following: # Audit logging for ResourceManager rm.audit.logger=${hadoop.root.logger} log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger=${rm.audit.logger} log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger=false #log4j.appender.RMAUDIT=org.apache.log4j.DailyRollingFileAppender log4j.appender.RMAUDIT.File=${yarn.log.dir}/rm-audit.log log4j.appender.RMAUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.RMAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n #log4j.appender.RMAUDIT.DatePattern=.yyyy-MM-dd log4j.appender.RMAUDIT=org.apache.log4j.RollingFileAppender log4j.appender.RMAUDIT.MaxBackupIndex=5 log4j.appender.RMAUDIT.MaxFileSize=1MB and # Audit logging for NodeManager nm.audit.logger=${hadoop.root.logger} log4j.logger.org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger=${nm.audit.logger} log4j.additivity.org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger=false #log4j.appender.NMAUDIT=org.apache.log4j.DailyRollingFileAppender log4j.appender.NMAUDIT.File=${yarn.log.dir}/nm-audit.log log4j.appender.NMAUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.NMAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n #log4j.appender.NMAUDIT.DatePattern=.yyyy-MM-dd log4j.appender.NMAUDIT=org.apache.log4j.RollingFileAppender log4j.appender.NMAUDIT.MaxBackupIndex=5 log4j.appender.NMAUDIT.MaxFileSize=1MB and restart the YARN service now we want to see if rm-audit.log and nm-audit.log are rotated , but because we set 1MB in MaxFileSize it will take time until logs will be rotated -rw-r--r-- 1 yarn hadoop 4180 Aug 28 09:32 rm-audit.log is it possible to in force the logs to increase quickly to 1M and rotated as rm-audit.log.1 , and rm-audit.log.2 , etc so we can sure that our log4j configuration is correctly ? ... View more