Member since
03-06-2017
45
Posts
4
Kudos Received
0
Solutions
05-09-2018
07:06 PM
@Matt Thanks for reply. My point behind "Nifi Slow Response" is UI gives slow response in some conditions i.e. 1. When I open configuration model window of any processor. 2. When I click on apply button after configuration. 3. When I am going to open controller services to configure one of my controller. 4. When I reload my Nifi instance. In all the above cases sometimes it works fine and respond me in a second or two but sometime it take 20 sec to 1 min or more then 1 minute. Now I want to know that I have enough memory still I am facing this type of response. And I use my NIFI with KYLO so I wont able to disable my processor, Is there any other way to solve this problem? Max Timer Driven Thread Count is set to 20 Max Cron Driven Thread Count is set to 50. Thanks in advance
... View more
05-09-2018
07:06 PM
@Matt Thanks for reply. My point behind "Nifi Slow Response" is UI gives slow response in some conditions i.e. 1. When I open configuration model window of any processor. 2. When I click on apply button after configuration. 3. When I am going to open controller services to configure one of my controller. In all the above cases sometimes it works fine and respond me in a second or two but sometime it take 20 sec to 1 min or more then 1 minute. Now I want to know that I have enough memory still I am facing this type of response. And I use my NIFI with KYLO so I wont able to disable my processor, Is there any other way to solve this problem? Max Timer Driven Thread Count is set to 20 Max Cron Driven Thread Count is set to 50. Thanks in advance
... View more
05-09-2018
03:36 PM
Thanks Shu for quick response. According to you, I can disable all invalid processor but I won't able to disable all stopped processor as 90% stopped processor in my nifi are useful and I can run it on time to time. And there may be possibilities of increase in no. of processor. When I check memory usage using "system-diagnostics" rest api then it give me below output: {"systemDiagnostics":{"aggregateSnapshot":{"totalNonHeap":"831.29 MB","totalNonHeapBytes":871669760,"usedNonHeap":"796.05 MB","usedNonHeapBytes":834719376,"freeNonHeap":"35.24 MB","freeNonHeapBytes":36950384,"maxNonHeap":"-1 bytes","maxNonHeapBytes":-1,"totalHeap":"19.55 GB","totalHeapBytes":20992491520,"usedHeap":"8.25 GB","usedHeapBytes":8862073056,"freeHeap":"11.3 GB","freeHeapBytes":12130418464,"maxHeap":"19.55 GB","maxHeapBytes":20992491520,"heapUtilization":"42.0%","availableProcessors":8,"processorLoadAverage":10.91,"totalThreads":428,"daemonThreads":201,"uptime":"00:23:16.322","flowFileRepositoryStorageUsage":{"freeSpace":"314.39 GB","totalSpace":"1.43 TB","usedSpace":"1.12 TB","freeSpaceBytes":337573191680,"totalSpaceBytes":1572096000000,"usedSpaceBytes":1234522808320,"utilization":"79.0%"},"contentRepositoryStorageUsage":[{"identifier":"default","freeSpace":"314.39 GB","totalSpace":"1.43 TB","usedSpace":"1.12 TB","freeSpaceBytes":337573191680,"totalSpaceBytes":1572096000000,"usedSpaceBytes":1234522808320,"utilization":"79.0%"}],"provenanceRepositoryStorageUsage":[{"identifier":"default","freeSpace":"314.39 GB","totalSpace":"1.43 TB","usedSpace":"1.12 TB","freeSpaceBytes":337573191680,"totalSpaceBytes":1572096000000,"usedSpaceBytes":1234522808320,"utilization":"79.0%"}],"garbageCollection":[{"name":"G1 Young Generation","collectionCount":1471,"collectionTime":"00:05:30.811","collectionMillis":330811},{"name":"G1 Old Generation","collectionCount":0,"collectionTime":"00:00:00.000","collectionMillis":0}],"statsLastRefreshed":"20:06:22 IST","versionInfo":{"niFiVersion":"1.4.0","javaVendor":"Oracle Corporation","javaVersion":"1.8.0_77","osName":"Linux","osVersion":"3.10.0-514.el7.x86_64","osArchitecture":"amd64","buildTag":"nifi-1.4.0-RC2","buildTimestamp":"12/04/2017 19:55:41 IST"}}}} As you can notice that, I have provided max 20GB memory. So, I think this is enough memory to run nifi smoothly with 15k+ no. of processor. As there is no limit of no. of processor dragged on canvas then what can I don to run nifi smoothly with 15k+ or 20k+ processors. Please let me know as soon as possible. Thanks in advance.
... View more
05-09-2018
07:41 AM
Hello, I am using Nifi with maximum ram of 26 GB and when i run my flows, it uses 33 GB. I have used an API mentioned below, to track the status of all my flows, http://localhost/nifi-api/flow/status Here is my response from the API {"controllerStatus":{"activeThreadCount":3,"queued":"52,687 / 27.18 MB","flowFilesQueued":52687,"bytesQueued":28495782,"runningCount":127,"stoppedCount":9918,"invalidCount":3239,"disabledCount":153,"activeRemotePortCount":0,"inactiveRemotePortCount":0}} After some time it started responding very slow unless i restart it again.I am facing this problem again and again, and i am not able to run my flows properly.
... View more
Labels:
- Labels:
-
Apache NiFi
05-04-2018
06:32 AM
I have setup NCM with 3 nodes. Somehow one node disconnected or failed. Again I started that node But It shown error message that 'Failed to connect node to cluster because local flow is different than cluster flow'. For this I have deleted flow.xml.gz file from the failure node and start it. Then it's work fine. But I don't want to delete flow.xml.gz file. Is there any way that NIFI automatically delete flow.xml.gz file from failure node when I will start failure node in the cluster.
... View more
Labels:
- Labels:
-
Apache NiFi
04-24-2018
12:58 PM
Hello, Thanks for replying soon, It works fine. But I am getting one more problem with input port and remote process group.
I have GetFile processor & RemoteProcessGroup inside Group1 & when I am going to connect both using input ports then I am getting drop down of all inpurt ports which are pl aced in root canvas. But I want to connect with "Group1_Port1" input port that are also placed inside my "Group1" process group. But I am not getting this input port in drop down when I connect GetFile processor with remote process group. Is there any way to connect GetFile processor with RemoteProcessGroup with port that are placed inside Group1 not on root canvas. I have also captured screen shot of my flows, Please check & reply as soon as possible. Thank
... View more
04-24-2018
11:01 AM
1 Kudo
I have setup a NCM with 2 nodes and create a flow using GetFile & PutFile processor inside a process group and configure GetFile that it runs only on primary node. Now for distributing whatever flow file I gets from getfile processor I use remote process group inside my process group using a input port. But when I try to connect GetFile processor to remote process group it gives message that "Does not have any input port".
So, I place my input port on root and then connect again getfile processor to remote process group then it connected successfully. But what when I want to use my input port inside my process group, is there any way to connect getfile processor to remote process group using input port which I use inside my process group. For placing input port on root, I am unable to give upstream connection to PutFile processor. Because my PutFile processor reside inside process group and input port insde root. What can I do for this scenario, Please reply.
... View more
07-11-2017
08:01 AM
Sorry....I attached the wrong code...please find it here nodereadfrommongo.txt
... View more
07-10-2017
03:44 PM
I have created a custom processor and after defining some properties it is showing " not a supported property" error as shown in the picture .Can anybody help me with this?
... View more
Labels:
- Labels:
-
Apache NiFi
06-14-2017
10:48 AM
Hello, I am getting a scenario in my nifi flow in which I will have to keep some data globally in runtime for all processor that contained in my flow, so I can able to get that global data in any processor. Thankyou
... View more
Labels:
- Labels:
-
Apache NiFi