Support Questions

johnmteabo · ‎04-20-2017

I have a nifi instance with numerous processors that are functioning fine.

Except one RouteOnAttribute Processor that has a 90GB queue before it. The processor is on, and it has possible thread count of 10. It is not currently running any threads.

The nifi instance max thread count is 100 (event and timer driven), and the current number of threads running is 1, for a getsftp processor that is running.

Heap usage is 32% and the content and flowfile repos are at 2%.

The only thing that processes this queue is if I copy the route on attribute processor (and its attached putsftp processors) and move the queue over to the new copied RouteOnAttribute Processor and hit play.

Any ideas on this?

MattWho · ‎04-20-2017

@John T

What version of NiFi are you running?

Do any of the connections leading from the RouteOnAttribute processor have back pressure being applied? If you are running a new enough version the connection will be highlighted red when back pressure is being applied. If back pressure is being applied the source processor to that connection will not be allowed to run.

Can you share a screenshot of your dataflow?

If the above is not the case, have you checked your nifi-app.log for any errors? look for Out of Memory (OOM) ERRor for example.

Thanks,

Matt

View solution in original post

scott_thompson · ‎04-21-2017

If it was a hung thread, you wouldn't be able to redirect the queue by stopping the original RouteOnAttribute and pointing it to the new copy. You would have to cycle the NiFi instance first. After you redirect the queue to the new copy of RouteOnAttribute, does it eventually stop processing as well? Could you write down and type up the expression here so we can have a look at it please?

johnmteabo · ‎04-21-2017

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this....

Check out my new question on how to change the default value for back-pressure.

https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold....

mpayne · ‎04-21-2017

@John T What are your settings in the Scheduling tab of the processor? I.e., if you right-click the processor and go to Configure, then click the Scheduling tab, is the Scheduling Strategy set to "Timer Driven"? What is the "Run Schedule" set to?

johnmteabo · ‎04-21-2017

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this....

Check out my new question on how to change the default value for back-pressure.

https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold....

Cloudera Community

Support Questions

RouteOnAttribute Processor will not Process data