I can find and read the files from inside the nifi docker container. Not sure why ListFile cannot: nifi@4f614c6c67c4:/data/sftp/cdr$ ls -1t | head -5 | egrep '^c.r' cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183 cdr_AP4-SYD-CCP-UCM_05_201905281913_53152 cdr_AP2-HKG-CCP-UCM_13_201905281912_89066 cdr_AP9-SYD-UCCE-UCM_07_201905281911_196358 cdr_AP8-HKG-UCCE-UCM_02_201905281910_243229 nifi@4f614c6c67c4:/data/sftp/cdr$ tail -n 5 cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183 1,8,973062,138959080,1559070286,8,0,0,"","",0,0,4,0,0,0,0,0,0,0,0,0,0,"0","0",138959081,3,0,1054035722,"b003864986001","b003864986001","",0,0,4,0,0,0,0,0,0, ... View more

I can see and read the files from inside the nifi container. Not sure why the nifi ListFile process isn't picking them up. nifi@4f614c6c67c4:/data/sftp/cdr$ ls -1t | head -5 | egrep '^c.r' cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183 cdr_AP4-SYD-CCP-UCM_05_201905281913_53152 cdr_AP2-HKG-CCP-UCM_13_201905281912_89066 cdr_AP9-SYD-UCCE-UCM_07_201905281911_196358 cdr_AP8-HKG-UCCE-UCM_02_201905281910_243229 nifi@4f614c6c67c4:/data/sftp/cdr$ tail -n 5 cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183 1,8,973062,138959080,1559070286,8,0,0,"","",0,0,4,0,0,0,0,0,0,0,0,0,0,"0","0",138959081,3,0,1054035722,"b003864986001","b003864986001","",0,0,4,0,0,0,0,0,0,0, ... View more

I'm starting to trial NiFi as a data ingestion engine. I would like to input the following datatypes: 1) collectd (UDP): I don't think that NiFi has a collectd parser, so I will need to direct these raw UDP streams to a locally running Telegraf and Logstash instance for parsing. 2) Syslog (UDP): I would like to experiment with routing raw syslog UDP packets (unprocessed) to destinations as well as filtering/parsing the syslog data using the NiFi syslog modules. 3) Netflow (UDP): I would like to take a heavy raw Netflow stream and test performance to forward only a subset of Netflow data based on a list of protocol types that I'm interested in (mapped against one of the Netflow data key values). What I haven't been able to understand in the documentation is how I can redirect raw UDP packet flows, (listening and then forwarding to two destinations), without having to process the particular data in the UDP packet. ... View more

Thanks @Matt Clarke. I guess I have a bit of reading to do to find the right processor that just routes raw UDP packets. ... View more