Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar

Introduction

ControllerStatusReportingTask Logs the 5-minute stats that are shown in the NiFi Summary Page for Processors and Connections. By Default, when configured and started it goes directly to nifi-app.log. These can be configured in the NiFi logging configuration to log to different files, here I try to describe steps to log it to a separate log file with Ambari.

Prerequisite

1) Assuming you already have HDF-2.x Installed on your VM/Server, Ambari, NiFi is up and running. If not, I would recommend "Ease of Deployment" section of this article to install it [You can also follow this article for Automated installation of HDF cluster or refer hortonworks.com for detailed steps]

Configuring "Advanced nifi-node-logback-env" section in Ambari

1. Navigate to your browser window and type in URL for Ambari as below and login to Ambari UI [UI is accessible at port 8080]

http://<YOUR_IP>:8080/ 

2. Once logged in click on NiFi service option on left side, click on “Configs” and expand “Advanced nifi-node-logback-env” section in configs and edit “logback.xml” template and add below lines just before last line </configuration>

<appender name="5MINUTES_FILE">
<file>${org.apache.nifi.bootstrap.config.log.dir}/5minutesStatistics.log</file>
<rollingPolicy>            
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/5minutesStatistics_%d.log</fileNamePattern>
<maxHistory>5</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<logger name="org.apache.nifi.controller.ControllerStatusReportingTask" level="INFO" additivity="false">
<appender-ref ref="5MINUTES_FILE" />
</logger>

11757-ambari-nifi-conf.png

3. Once above lines for creating a new file named 5minutesStatistics.log to save all the 5Minutes statistics details the Reporting task creates, click save and enter details of what configuration is changed.

4. Once saved, Ambari will suggest restart of NiFi service, click restart (It might take up to 2 minutes to complete restart and NiFi UI to come online)

Configuring ControllerStatusReportingTask in NiFi

1. Once NiFi is restarted, navigate to NiFi User Interface on any node and click on the ‘Controller Settings’ tab on right top corner. A window like below will popup. Select “Reporting Tasks” tab Click on the ‘+’ on right corner, when a selection is requested click on ‘ControllerStatusReportingTask’

11759-2controller-add.jpg

2. Once selected click add, the Reporting task will be in stopped state, Now you may click start:

11760-3controller-view.jpg

3. Once started, the 5minutes statistics would have started logging onto 5minutesStatistics.log

11761-4controller-start.jpg

Verifying Log created in the Server

1. ControllerStatusReportingTask will log the 5-minute stats that are shown in the NiFi Summary Page for Processors and Connections to 5minutesStatistics.log. Let’s verify the same:

# tail -f  /var/log/nifi/5minutesStatistics.log

11762-5view-log.jpg

2. ControllerStatusReportingTask started logging the 5-min processor status to the log we specified and it will be rolled as per the configuration we provided.

3. Once verified, stop the Controller service.

Hope this Helps..

Thanks,

Jobin George


controller.jpg
4,522 Views
Comments
avatar
Contributor

Hi,Jobin George!

I tried this method but it doesn't works. A separate log file is not even created. I've all ready tried many more modifications of this configuration but still nothing. What I am doing wrong? The key moment is - "A separate log file is not even created".

It is confusing what triggers this task to run. Do you have any additional info on that or know if there is any way to configure it more precisely?

Version history
Last update:
‎08-17-2019 05:23 AM
Updated by:
Contributors