Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

RouteOnAttribute Processor will not Process data

avatar
Rising Star

I have a nifi instance with numerous processors that are functioning fine.

Except one RouteOnAttribute Processor that has a 90GB queue before it. The processor is on, and it has possible thread count of 10. It is not currently running any threads.

The nifi instance max thread count is 100 (event and timer driven), and the current number of threads running is 1, for a getsftp processor that is running.

Heap usage is 32% and the content and flowfile repos are at 2%.

The only thing that processes this queue is if I copy the route on attribute processor (and its attached putsftp processors) and move the queue over to the new copied RouteOnAttribute Processor and hit play.

Any ideas on this?

1 ACCEPTED SOLUTION

avatar
Super Mentor
@John T

What version of NiFi are you running?

Do any of the connections leading from the RouteOnAttribute processor have back pressure being applied? If you are running a new enough version the connection will be highlighted red when back pressure is being applied. If back pressure is being applied the source processor to that connection will not be allowed to run.

Can you share a screenshot of your dataflow?

If the above is not the case, have you checked your nifi-app.log for any errors? look for Out of Memory (OOM) ERRor for example.

Thanks,

Matt

View solution in original post

13 REPLIES 13

avatar
New Contributor

If it was a hung thread, you wouldn't be able to redirect the queue by stopping the original RouteOnAttribute and pointing it to the new copy. You would have to cycle the NiFi instance first. After you redirect the queue to the new copy of RouteOnAttribute, does it eventually stop processing as well? Could you write down and type up the expression here so we can have a look at it please?

avatar
Rising Star

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this....

Check out my new question on how to change the default value for back-pressure.

https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold....

avatar
Expert Contributor

@John T What are your settings in the Scheduling tab of the processor? I.e., if you right-click the processor and go to Configure, then click the Scheduling tab, is the Scheduling Strategy set to "Timer Driven"? What is the "Run Schedule" set to?

avatar
Rising Star

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this....

Check out my new question on how to change the default value for back-pressure.

https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold....