About johnmteabo

johnmteabo · ‎08-07-2017

ok great! thank you!!!

johnmteabo · ‎08-07-2017

In apache nifi how can I make the putfile processor use . rename as it writes a file out to the filesystem? or does it do this by default? Where it would write the file as a .filename and then rename it to the actual filename? John

johnmteabo · ‎04-21-2017

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this.... Check out my new question on how to change the default value for back-pressure. https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold.html

johnmteabo · ‎04-21-2017

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this.... Check out my new question on how to change the default value for back-pressure. https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold.html

johnmteabo · ‎04-21-2017

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this.... Check out my new question on how to change the default value for back-pressure. https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold.html

johnmteabo · ‎04-21-2017

Thanks for your help on this! It turns out there was back-pressure issue on later processors. The new default of NiFi 1.0+ is 1GB. I was just made aware of this.... Check out my new question on how to change the default value for back-pressure. https://community.hortonworks.com/questions/97788/change-default-connection-back-pressure-threshold.html

johnmteabo · ‎04-20-2017

@Matt Clarke Thanks for the quick response! we are using 1.1.1. None of the processors before or after have any back-pressure applied. I also checked my nifi-app.log and did not see any OOMs, I checked our heap and it was at 30 some percent when I checked. The whole nifi-app.log actually outputs nothing when this happens. As I told Bryan, this is on a network that can only receive data, no data can come out, so getting a screenshot would be hard, but its basically 1 input queue to a route on attribute proccessor that is routing on 1 attrbiute and has about 40 possible routes. John

johnmteabo · ‎04-20-2017

I am using Apache nifi 1.1.1. I did this but the problem is It's on a network that I can only send data into. I can't take any data out. I see a bunch of Event Driven Process thread-92 id=89206 Time_waiting Anything in particular we should be looking for? John

johnmteabo · ‎04-20-2017

I have a nifi instance with numerous processors that are functioning fine. Except one RouteOnAttribute Processor that has a 90GB queue before it. The processor is on, and it has possible thread count of 10. It is not currently running any threads. The nifi instance max thread count is 100 (event and timer driven), and the current number of threads running is 1, for a getsftp processor that is running. Heap usage is 32% and the content and flowfile repos are at 2%. The only thing that processes this queue is if I copy the route on attribute processor (and its attached putsftp processors) and move the queue over to the new copied RouteOnAttribute Processor and hit play. Any ideas on this?

johnmteabo · ‎04-17-2017

@mqureshi That's not quite what I'm looking for. I know the the filename on the REMOTE system contains file123.txt. There may be any nubmber of random characters of numbers preceding or proceeding "file123.txt" Adding contains wont work because I need the remote file fetch to get all files that have file123.txt in the filename. Not have nifi look to see if the attribute contains something. John

Online	Offline
Last Visited	‎03-04-2019 02:48 PM

Member Since	‎10-08-2016 04:38 PM
Last Visited	‎03-04-2019 02:48 PM
Posts	59
Kudos received	16

Cloudera Community

Re: nifi putfile dot rename?

nifi putfile dot rename?

Re: RouteOnAttribute Processor will not Process da...

Re: RouteOnAttribute Processor will not Process da...

Re: RouteOnAttribute Processor will not Process da...

Re: RouteOnAttribute Processor will not Process da...

Re: RouteOnAttribute Processor will not Process da...

Re: RouteOnAttribute Processor will not Process da...

RouteOnAttribute Processor will not Process data

Re: Does the FetchSftp Processor support wildcards...