- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
RouteOnAttribute Processor will not Process data
- Labels:
-
Apache NiFi
Created ‎04-20-2017 07:24 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a nifi instance with numerous processors that are functioning fine.
Except one RouteOnAttribute Processor that has a 90GB queue before it. The processor is on, and it has possible thread count of 10. It is not currently running any threads.
The nifi instance max thread count is 100 (event and timer driven), and the current number of threads running is 1, for a getsftp processor that is running.
Heap usage is 32% and the content and flowfile repos are at 2%.
The only thing that processes this queue is if I copy the route on attribute processor (and its attached putsftp processors) and move the queue over to the new copied RouteOnAttribute Processor and hit play.
Any ideas on this?
Created ‎04-20-2017 07:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What version of NiFi are you running?
Do any of the connections leading from the RouteOnAttribute processor have back pressure being applied? If you are running a new enough version the connection will be highlighted red when back pressure is being applied. If back pressure is being applied the source processor to that connection will not be allowed to run.
Can you share a screenshot of your dataflow?
If the above is not the case, have you checked your nifi-app.log for any errors? look for Out of Memory (OOM) ERRor for example.
Thanks,
Matt
Created ‎04-21-2017 03:35 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If it was a hung thread, you wouldn't be able to redirect the queue by stopping the original RouteOnAttribute and pointing it to the new copy. You would have to cycle the NiFi instance first. After you redirect the queue to the new copy of RouteOnAttribute, does it eventually stop processing as well? Could you write down and type up the expression here so we can have a look at it please?
Created ‎04-21-2017 05:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this....
Check out my new question on how to change the default value for back-pressure.
Created ‎04-21-2017 03:57 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@John T What are your settings in the Scheduling tab of the processor? I.e., if you right-click the processor and go to Configure, then click the Scheduling tab, is the Scheduling Strategy set to "Timer Driven"? What is the "Run Schedule" set to?
Created ‎04-21-2017 05:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this....
Check out my new question on how to change the default value for back-pressure.

- « Previous
-
- 1
- 2
- Next »