Introduction
ControllerStatusReportingTask Logs the 5-minute stats that are shown in the NiFi Summary Page for Processors and Connections. By Default, when configured and started it goes directly to nifi-app.log. These can be configured in the NiFi logging configuration to log to different files, here I try to describe steps to log it to a separate log file with Ambari.
Prerequisite
1) Assuming you already have HDF-2.x Installed on your VM/Server, Ambari, NiFi is up and running. If not, I would recommend "Ease of Deployment" section of this article to install it [You can also follow this article for Automated installation of HDF cluster or refer hortonworks.com for detailed steps]
Configuring "Advanced nifi-node-logback-env" section in Ambari
1. Navigate to your browser window and type in URL for Ambari as below and login to Ambari UI [UI is accessible at port 8080]
http://<YOUR_IP>:8080/
2. Once logged in click on NiFi service option on left side, click on “Configs” and expand “Advanced nifi-node-logback-env” section in configs and edit “logback.xml” template and add below lines just before last line </configuration>
<appender name="5MINUTES_FILE">
<file>${org.apache.nifi.bootstrap.config.log.dir}/5minutesStatistics.log</file>
<rollingPolicy>
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/5minutesStatistics_%d.log</fileNamePattern>
<maxHistory>5</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<logger name="org.apache.nifi.controller.ControllerStatusReportingTask" level="INFO" additivity="false">
<appender-ref ref="5MINUTES_FILE" />
</logger>
3. Once above lines for creating a new file named 5minutesStatistics.log to save all the 5Minutes statistics details the Reporting task creates, click save and enter details of what configuration is changed.
4. Once saved, Ambari will suggest restart of NiFi service, click restart (It might take up to 2 minutes to complete restart and NiFi UI to come online)
Configuring ControllerStatusReportingTask in NiFi
1. Once NiFi is restarted, navigate to NiFi User Interface on any node and click on the ‘Controller Settings’ tab on right top corner. A window like below will popup. Select “Reporting Tasks” tab Click on the ‘+’ on right corner, when a selection is requested click on ‘ControllerStatusReportingTask’
2. Once selected click add, the Reporting task will be in stopped state, Now you may click start:
3. Once started, the 5minutes statistics would have started logging onto 5minutesStatistics.log
Verifying Log created in the Server
1. ControllerStatusReportingTask will log the 5-minute stats that are shown in the NiFi Summary Page for Processors and Connections to 5minutesStatistics.log. Let’s verify the same:
# tail -f /var/log/nifi/5minutesStatistics.log
2. ControllerStatusReportingTask started logging the 5-min processor status to the log we specified and it will be rolled as per the configuration we provided.
3. Once verified, stop the Controller service.
Hope this Helps..
Thanks,
Jobin George
