Member since
10-15-2017
14
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4380 | 10-25-2017 10:13 PM |
12-16-2017
05:14 AM
Unfortunately, the validation query doesn't appear to be working. We are still gettng the same issue, the putsql's lose the connection to oracle(looking in oracle it doesn't show any connections), but the controller in nifi, is showing connected, so the putsql's just stop.
... View more
10-25-2017
10:13 PM
We have resolved the issue. As usual in hindsight it seems obvious. One of the processes, a transformxml, had the thread count set at 20. This seemed fine as most files going through were about 3m each. Our Java heap was set at 64g. However new files were introduced, that were in the 3g range. Simple math, 3 x 20=60g. used up for that 1 process. Since there was usually about 200k files flowing through at any one time, memory quickly got exhausted. We put a route attribute, to route these large files to a process that only has a couple of threads, while the smaller files could still filter through the process with 20 threads. Thanks for all the help suggestions. @Hans Feldmann
... View more
10-23-2017
09:17 PM
Thanks, but i see that polling is mostly for input files,and that doesn't seem to be an issue, Most times there is nothing in the input directory. but still unresponsive.
... View more
10-18-2017
12:51 AM
Thanks, but we had the heap set to 64g. I tried downsizing to 32g. Memory on machine is 80g I will see how that works.Usually putting through about 50000 files every 1/2 hour. # JVM memory settings
java.arg.2=-Xms32g
java.arg.3=-Xmx32g
... View more
10-17-2017
04:13 PM
nifi flow is unresponsive. did nifi.sh status. 2017-10-17 12:09:15,040 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi is running at PID 24007 but is not responding to ping requests
... View more
Labels:
- Labels:
-
Apache NiFi
10-15-2017
02:29 PM
thanks, I will give it a try.
... View more
10-15-2017
01:58 PM
Our flow files ends with putsql's to oracle database. At random times, the flow stops with files in the queue before the putsql's. (5 minute activity shows 0). Stopping and starting nifi does not do anything. However we found that disabling and enabling the dbcpconnection pool starts it going again. We are on nifi.1.2.0
... View more
Labels:
- Labels:
-
Apache NiFi