About jonathan_bell

jonathan_bell · ‎12-10-2018

example of flow, the routeonattribute is started, but nothing going through it, even though queue has many flowfiles.

jonathan_bell · ‎12-10-2018

After updating to 1.8, Random processes stop accepting data. There is data in the queue, the process is started, but no activity go through it. If I stop and start the process, that seems to trigger it to start accepting flowfiles.

jonathan_bell · ‎06-28-2018

ok.thanks for the clarification. however the modification date seems to be of the flowfile, not the actual file. I tried a file that has a modification date of 6 months ago, but the modified date is current date.

jonathan_bell · ‎06-27-2018

We have many historical number of files on server, but only want to begin listing files from a current date. Right now it appears that we can only clear the state,which starts the listing at the earliest dated file. Nifi version 1.5.0

jonathan_bell · ‎06-04-2018

Thanks for the feedback. I have read that description, and that is exactly what we want it to do. However even though there are files on the server that have a timestamp newer than what is listed in the Listsftp state, it still does not read them.

jonathan_bell · ‎06-04-2018

We have listsftp/fetchsftp process configured on a cluster. They work great, however the Listsftp will only fire once after started even though we have it scheduled for 30 minute interval. It doesn't list any files after the first time it starts, even though there are new files added to directory every 30 minutes. We are using nifi-1.5.0

jonathan_bell · ‎12-16-2017

Unfortunately, the validation query doesn't appear to be working. We are still gettng the same issue, the putsql's lose the connection to oracle(looking in oracle it doesn't show any connections), but the controller in nifi, is showing connected, so the putsql's just stop.

jonathan_bell · ‎10-25-2017

We have resolved the issue. As usual in hindsight it seems obvious. One of the processes, a transformxml, had the thread count set at 20. This seemed fine as most files going through were about 3m each. Our Java heap was set at 64g. However new files were introduced, that were in the 3g range. Simple math, 3 x 20=60g. used up for that 1 process. Since there was usually about 200k files flowing through at any one time, memory quickly got exhausted. We put a route attribute, to route these large files to a process that only has a couple of threads, while the smaller files could still filter through the process with 20 threads. Thanks for all the help suggestions. @Hans Feldmann

jonathan_bell · ‎10-23-2017

Thanks, but i see that polling is mostly for input files,and that doesn't seem to be an issue, Most times there is nothing in the input directory. but still unresponsive.

jonathan_bell · ‎10-18-2017

Thanks, but we had the heap set to 64g. I tried downsizing to 32g. Memory on machine is 80g I will see how that works.Usually putting through about 50000 files every 1/2 hour. # JVM memory settings java.arg.2=-Xms32g java.arg.3=-Xmx32g

jonathan_bell · ‎10-17-2017

nifi flow is unresponsive. did nifi.sh status. 2017-10-17 12:09:15,040 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi is running at PID 24007 but is not responding to ping requests

jonathan_bell · ‎10-15-2017

thanks, I will give it a try.

jonathan_bell · ‎10-15-2017

Our flow files ends with putsql's to oracle database. At random times, the flow stops with files in the queue before the putsql's. (5 minute activity shows 0). Stopping and starting nifi does not do anything. However we found that disabling and enabling the dbcpconnection pool starts it going again. We are on nifi.1.2.0

Online	Offline
Last Visited	‎01-03-2019 10:18 PM

Date Registered	‎10-15-2017 01:37 PM
Last Visited	‎01-03-2019 10:18 PM
Total Messages Posted	14

Re: NIFI not responding.

Re: Process stops accepting flow files.

Process stops accepting flow files.

Re: How to set specific time for listsftp to start...

How to set specific time for listsftp to start.

Re: Listsftp only runs first time after start

Listsftp only runs first time after start

Re: dbcpconnection pool drops connection.

Re: NIFI not responding.

Re: NIFI not responding.

Re: NIFI not responding.

NIFI not responding.

Re: dbcpconnection pool drops connection.

dbcpconnection pool drops connection.