Support Questions

Find answers, ask questions, and share your expertise

about - NiFi Monitoring - ControllerStatusReportingTask with Ambari

Contributor

Hi guys!

Please, see this tutorial - https://community.hortonworks.com/articles/79849/nifi-monitoring-controllerstatusreportingtask-with.... before you read my question. ()

I tried this method but it doesn't works. A separate log file is not even created. I've all ready tried many more modifications of this configuration but still nothing. What I am doing wrong? The key moment is - "A separate log file is not even created".

PS

I've already asked this question inside this tutorial but still I have no answer.

Thanks for help!

20 REPLIES 20

Hi @Alex Coast

I followed the same tutorial and had issues getting the file generated too, initially.
Had to make a few minor adjustments, and following works well for me;

<appender name="5MINUTES_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> 
<file>${org.apache.nifi.bootstrap.config.log.dir}/5minutesStatistics.log</file> 
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> 
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/5minutesStatistics_%d.log</fileNamePattern> 
<maxHistory>5</maxHistory> 
</rollingPolicy> 
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> 
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern> 
</encoder> 
</appender> 
<logger name="org.apache.nifi.controller.ControllerStatusReportingTask" level="INFO" additivity="false"> 
<appender-ref ref="5MINUTES_FILE" /> 
</logger> 

PS. If the above helps you to resolve your issue please take a moment to log in and mark the answer as accepted 🙂

Contributor

Hi @Jonathan Sneep. Thanks for your answer, but I've allready did like that but still nothing.

My steps:

1. Add the config to logback.xml

2. Restart service nifi

3. Do something in NiFi (like start some dataflow)

And the result is nothing - No new log file with separate logs which I configure

What I'm doing wrong?

Master Guru

@Alex Coast

What version of NiFi are you running?

Thanks,

Matt

@Alex Coast

What version of HDP/HDF are you using?
And just to be sure, you are adding this via Ambari right? Under Advanced nifi-node-logback-env of the NiFi config?
If you share your HDP/HDF version and your current nifi-node-logback-env (logback.xml) I will try to reproduce your scenario exactly 🙂

Contributor

@Jonathan Sneep

No, via Ambari it's too simple :))) I'm trying via command line hard code 🙂 Maybe that's why it doesn't work. If so how can I configure logback.xml by my method (via cli) ? Thanks!

Contributor

I have no any Ambari that's why I need to do this configuration without it. Is anybody know how to do this right?

Master Guru

@Alex Coast

-

I just enabled what you are trying to do using Apache NiFi 1.8 without any issue.

             <appender name="STATUS_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
            <file>/var/log/nifi/nifi-status.log</file>
            <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!--
            For daily rollover, use 'user_%d.log'.
            For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
            To GZIP rolled files, replace '.log' with '.log.gz'.
            To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>/var/log/nifi/nifi-status_%d.log</fileNamePattern>
            <!-- keep 30 log files worth of history -->
            <maxHistory>2</maxHistory>
            </rollingPolicy>
            <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
            </encoder>
            </appender>

-

I then added separate loggers for for Connections and Processors

            <logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Connections" level="INFO" additivity="false">
            <appender-ref ref="STATUS_FILE"/>
            </logger>
            <logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Processors" level="INFO" additivity="false">
            <appender-ref ref="STATUS_FILE"/>
            </logger>

-

I successfully see my nifi-status.log being created and containing proper output from these loggers.

-

1. Double check your logback.xml for any formatting mistakes. If a formatting mistake is found, you will need to restart nifi after fixing that formatting mistake. For example: forgetting to close a code block like "<logger" without corresponding "</logger>".

2. Make sure you do not have any comment blocks that were started and not properly ended.
This starts a comment block: <!--

This ends that comment block: -->

3. Make sure there are no comment blocks started within existing comment blocks.

4. Make sure the user running your NiFi service has permissions to create log in configured directory.

-

93267-screen-shot-2018-11-15-at-95240-am.png

Thank you,

Matt

Contributor

Hi @Matt Clarke !

Thank you for your detailed answer. I tried 3 or 4 different ways to configure this logs! And yes?I always check from 5 to 10 times my configs before restart nifi service.

Here is my permissions and config(bottom):

I still have no any file with this configured separate status logs.

93634-logback1.png

93635-logback2.png

           <!-- Test configuration for ControllerStatusReportingTask 2018-11-07 by AL -->


    <appender name="STATUS_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-status.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>/var/log/nifi/nifi-status_%d.log</fileNamePattern>
            <!-- keep 5  log files worth of history -->
            <maxHistory>5</maxHistory>
        </rollingPolicy>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>



    <!-- Logger to redirect controller status/reporting task Connections output to a different file. -->
    <logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Connections" level="INFO" additivity="false">
        <appender-ref ref="STATUS_FILE"/>
    </logger>
    <logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Processors" level="INFO" additivity="false">
        <appender-ref ref="STATUS_FILE"/>
    </logger> 

Master Guru

@Alex Coast

You screenshot indicates you have .swp file associated to your logback.xml file. When using vi for example to edit the logback.xml file, this swp file is created to hold you changes until you exit vi. I see that both your swap file and the original logback.xml are very different in sizes. Appears as though your NiFi may still be using the logback.xml which does not contian your changes that exist in the swp file only.

-

Thank you,

Matt

Contributor

@Matt Clarke Nope.Still nothing. And YES I've restarted the nifi service and check the logback.xml for actual config.

93641-logback1.png

93642-logback2.png

Master Guru

If you upload your logback.xml here, I would be happy to take a look.

Contributor

Contributor

Contributor

Hi @Matt Clarke !

I did like you advised (removing or commenting out the STATUS_FILE appender

and just have your loggers use the default appender (nifi-app.log) to see if any output is created.) and I don't I understand what kind of new info I must to see? But at the beginning when I tried to configure these separate Connections logs I thought that I could monitoring the queues by those -> but today when I saw this Connections logs (before that I created some queue issue) I don't get it - where in this logs I can find some kind of value that tells me something about my queue issue. I searched, grep by Connector ID and etc - but I couldn't find anything.

My version of NIFI is nifi-1.7.1-RC1

Master Guru

@Alex Coast:1

I see no issues with your logback.xml.

Is it possible that Ansible is removing the the log as soon as it is created.

-

Try removing or commenting out the STATUS_FILE appender
and just have your loggers use the default appender (nifi-app.log) to see if any output is created.

<logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Connections" level="INFO"/>

<logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Processors" level="INFO"/>

-

Make sure your reporting task is running (adjust the run schedule so it runs more frequently).

Sorry to say this feels more environmental at this point.
The configuration is working correctly in my NiFi 1.8.0 just fine.

-

Master Guru

@Alex Coast

-

It would be very hard to miss the output produced by these loggers. Both output a table:

93310-screen-shot-2018-11-19-at-102148-am.png

93311-screen-shot-2018-11-19-at-102222-am.png

Depending on number of processors and connections you have on your canvas, these tables can easily get very long.

-

You can also use the NIFi rest-api to return all the same information and then some. The returned JSPN could be parsed for things like percentusage to see if backpresure is being applied anyway, checking FlowFiles(in) vs FlowFiles(out stats on components to look for outage or decrease/increase in volumes, etc....

-

curl 'http://<nifi-hostname>:<nifi-port>/nifi-api/flow/process-groups/root/status?recursive=true'

-

Thank you,

Matt

Contributor

@Matt Clarke Nope.Still nothing. And YES I've restarted the nifi service and check the logback.xml for actual config

Contributor
cat logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at
      http://www.apache.org/licenses/LICENSE-2.0
  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->


<configuration scan="true" scanPeriod="30 seconds">
    <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
        <resetJUL>true</resetJUL>
    </contextListener>


    <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <!--
              For daily rollover, use 'app_%d.log'.
              For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
            <maxFileSize>100MB</maxFileSize>
            <!-- keep 30 log files worth of history -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <immediateFlush>true</immediateFlush>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>


    <appender name="USER_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!--
              For daily rollover, use 'user_%d.log'.
              For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user_%d.log</fileNamePattern>
            <!-- keep 30 log files worth of history -->
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>


    <appender name="BOOTSTRAP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!--
              For daily rollover, use 'user_%d.log'.
              For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap_%d.log</fileNamePattern>
            <!-- keep 5 log files worth of history -->
            <maxHistory>5</maxHistory>
        </rollingPolicy>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>




           <!-- Test configuration for ControllerStatusReportingTask 2018-11-07 by AL -->


    <appender name="STATUS_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-status.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-status_%d.log</fileNamePattern>
            <!-- keep 5  log files worth of history -->
            <maxHistory>5</maxHistory>
        </rollingPolicy>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>




    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>




    <!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->


    <logger name="org.apache.nifi" level="INFO"/>
    <logger name="org.apache.nifi.processors" level="WARN"/>
    <logger name="org.apache.nifi.processors.standard.LogAttribute" level="INFO"/>
    <logger name="org.apache.nifi.processors.standard.LogMessage" level="INFO"/>
    <logger name="org.apache.nifi.controller.repository.StandardProcessSession" level="WARN" />




    <logger name="org.apache.zookeeper.ClientCnxn" level="ERROR" />
    <logger name="org.apache.zookeeper.server.NIOServerCnxn" level="ERROR" />
    <logger name="org.apache.zookeeper.server.NIOServerCnxnFactory" level="ERROR" />
    <logger name="org.apache.zookeeper.server.quorum" level="ERROR" />
    <logger name="org.apache.zookeeper.ZooKeeper" level="ERROR" />
    <logger name="org.apache.zookeeper.server.PrepRequestProcessor" level="ERROR" />


    <logger name="org.apache.calcite.runtime.CalciteException" level="OFF" />


    <logger name="org.apache.curator.framework.recipes.leader.LeaderSelector" level="OFF" />
    <logger name="org.apache.curator.ConnectionState" level="OFF" />


    <!-- Logger for managing logging statements for nifi clusters. -->
    <logger name="org.apache.nifi.cluster" level="INFO"/>


    <!-- Logger for logging HTTP requests received by the web server. -->
    <logger name="org.apache.nifi.server.JettyServer" level="INFO"/>


    <!-- Logger for managing logging statements for jetty -->
    <logger name="org.eclipse.jetty" level="INFO"/>


    <!-- Suppress non-error messages due to excessive logging by class or library -->
    <logger name="org.springframework" level="ERROR"/>


    <!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) -->
    <logger name="org.glassfish.jersey.internal.Errors" level="ERROR"/>


    <!--
        Logger for capturing user events. We do not want to propagate these
        log events to the root logger. These messages are only sent to the
        user-log appender.
    -->
    <logger name="org.apache.nifi.web.security" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.web.api.config" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.authorization" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.cluster.authorization" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.web.filter.RequestLogger" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>




    <!--
        Logger for capturing Bootstrap logs and NiFi's standard error and standard out.
    -->
    <logger name="org.apache.nifi.bootstrap" level="INFO" additivity="false">
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>
    <logger name="org.apache.nifi.bootstrap.Command" level="INFO" additivity="false">
        <appender-ref ref="CONSOLE" />
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>


    <!-- Everything written to NiFi's Standard Out will be logged with the logger org.apache.nifi.StdOut at INFO level -->
    <logger name="org.apache.nifi.StdOut" level="INFO" additivity="false">
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>


    <!-- Everything written to NiFi's Standard Error will be logged with the logger org.apache.nifi.StdErr at ERROR level -->
    <logger name="org.apache.nifi.StdErr" level="ERROR" additivity="false">
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>


    <!-- Logger to redirect controller status/reporting task Connections output to a different file. -->
    <logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Connections" level="INFO" additivity="false">
        <appender-ref ref="STATUS_FILE"/>
    </logger>
    <logger name="org.apache.nifi.controller.ControllerStatusReportingTask.Processors" level="INFO" additivity="false">
        <appender-ref ref="STATUS_FILE"/>
    </logger>




    <root level="INFO">
        <appender-ref ref="APP_FILE"/>
    </root>


</configuration>


Contributor

Hi @Matt Clarke !

I did like you advised (removing or commenting out the STATUS_FILE appender

and just have your loggers use the default appender (nifi-app.log) to see if any output is created.) and I don't I understand what kind of new info I must to see? But at the beginning when I tried to configure these separate Connections logs I thought that I could monitoring the queues by those -> but today when I saw this Connections logs (before that I created some queue issue) I don't get it - where in this logs I can find some kind of value that tells me something about my queue issue. I searched, grep by Connector ID and etc - but I couldn't find anything.

My version of NIFI is nifi-1.7.1-RC1

Contributor

@Matt Clarke Yes, I see these logs in my nifi-app.log, but I don't need some changes in default logback.xml for this 🙂 Maybe it's all about NiFi version. When we'll migrate from NiFi 1.7.1 to 1.8.0 I try to do this task again 🙂 Thank you Matt!

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.