@Matt Clarke Can you explain.. how we can ensure the remaining threads was using by other components Ex :controller services / Reporting tasks. If you explain that really helpful to close this thread. ... View more

@Matt Clarke How we can ensure that ? the remaining threads was using by other Ex :controller services / Reporting tasks. ... View more

@Matt Clarke Recently we didn't do any update and we have tried with restart also but still we are seeing differences in the thread values in status bar. What actually we configured. What actually running. How this drastic differences ? pls help to close this. ... View more

We are running only two processor group and the overall usage you can see in the 2 PG ( 8 + 4). Hdf - 3.0.1.0-43 Maximum Timer Driven Thread Count = 100 Max Event Driven Thread Count = 5 we have two nodes but it not configured as cluster. ... View more

We are running only two processor group and the overall usage you can see in the 2 PG ( 8 + 4). Hdf - 3.0.1.0-43 Maximum Timer Driven Thread Count = 100 Max Event Driven Thread Count = 5 we have two nodes but it not configured as cluster. ... View more

@Bryan Bende experts inputs required. ... View more

@Matt Clarke Hi Bro need your inputs. ... View more

Hi Abdelkrim, Thanks its working now The same we can do in standalone as well right? in rpg also it taking from one node? how we can process from multiple nodes and balance the load. Why we need to put input port in outside of processor group? root path. ... View more

@Abdelkrim Hadjidj Thanks if the new file comes it processing and it fetch to hdfs but i am getting the below error also in Fetchsftp. 2017-11-02 01:56:41,942 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.FetchSFTP FetchSFTP[id=788c7b35-015f-1000-0000-00000f3979fd] Failed to fetch content for StandardFlowFileRecord[uuid=af918b98-5632-4877-9876-221477204e73,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1509553663114-11, container=default, section=11], offset=677691, length=0],offset=0,name=a.txt,size=0] from filename /home/1548691/*.txt on remote host 10.20.174.137:22 due to java.io.IOException: Failed to obtain file content for /home/1548691/*.txt; routing to failure: java.io.IOException: Failed to obtain file content for /home/1548691/*.txt 2017-11-02 01:56:41,943 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.FetchSFTP java.io.IOException: Failed to obtain file content for /home/1548691/*.txt at org.apache.nifi.processors.standard.util.SFTPTransfer.getInputStream(SFTPTransfer.java:300) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.processors.standard.FetchFileTransfer.onTrigger(FetchFileTransfer.java:236) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_60] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_60] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_60] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] Caused by: com.jcraft.jsch.SftpException: /home/1548691/*.txt is not unique: [/home/1548691/b.txt, /home/1548691/HKLPATHAS03.txt, /home/1548691/a.txt] at com.jcraft.jsch.ChannelSftp.isUnique(ChannelSftp.java:2965) ~[jsch-0.1.54.jar:na] at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1314) ~[jsch-0.1.54.jar:na] at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1290) ~[jsch-0.1.54.jar:na] at org.apache.nifi.processors.standard.util.SFTPTransfer.getInputStream(SFTPTransfer.java:292) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] ... 13 common frames omitted @ how to process all set of files instead of .txt and it picking from only one node? how to pick from all 3 nodes ? ... View more