Support Questions
Find answers, ask questions, and share your expertise

Debugging NiFi flow

Solved Go to solution

Debugging NiFi flow

Hi,

I have a NiFi flow that consumes lot of memory. How can I diagnosis the flow to isolate the problem and find the processor that

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Debugging NiFi flow

Guru

You can use NiFis native reporting to monitor:

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Reporting_Tasks

For your needs, there is a memory monitor which checks the amount of Java Heap available in the JVM for a particular JVM Memory Pool. If the amount of space used exceeds some configurable threshold, will warn (via a log message and System-Level Bulletin) that the memory pool is exceeding this threshold.

See this post for more on NiFi Reporting:

https://community.hortonworks.com/questions/69004/nifi-monitoring-processor-and-nifi-service.html#an...

View solution in original post

3 REPLIES 3
Highlighted

Re: Debugging NiFi flow

Contributor
@Houssam Manik have you enabled debug logging for the processors that you are using to try and investigate further? Your configuration can be found at <NIFI_HOME>/conf/logback.xml, you can set processors etc to debug level to investigate further, eg:
<logger name="org.apache.nifi.processors.standard.GetFile" level="DEBUG"/>

Are you running with a default configuration? If so you could try bumping your memory settings in conf/bootstrap.conf. They default to 512mb, you may want to try bumping it up to 1024mb.

If you could post more about your flow/use case I'm sure we can help narrow down which processor is causing the problem.

Highlighted

Re: Debugging NiFi flow

Guru

You can use NiFis native reporting to monitor:

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Reporting_Tasks

For your needs, there is a memory monitor which checks the amount of Java Heap available in the JVM for a particular JVM Memory Pool. If the amount of space used exceeds some configurable threshold, will warn (via a log message and System-Level Bulletin) that the memory pool is exceeding this threshold.

See this post for more on NiFi Reporting:

https://community.hortonworks.com/questions/69004/nifi-monitoring-processor-and-nifi-service.html#an...

View solution in original post

Highlighted

Re: Debugging NiFi flow

Cloudera Employee

In Apache Nifi 1.1 there is also the new back pressure indicator bars that you can configure. These may be able to give you a clue as to where the bottleneck is...

Don't have an account?