Debugging NiFi Flows in Docker - problem with ListFile?

[#1]

Hi there,

I'm running NiFi in docker. I'm trying to enable flow debugs and also to determine if NiFi can actually see in my data directory path. I'm trying to use a simple recursive list file processor to find the files based on a regular expression "^c.r".

I can docker exec into the container and can see that the nifi container can "see" into my filepath OK:

nifi@4f614c6c67c4:/data/sftp/cdr$ ls -1t | head -5 | egrep '^c.r'
cdr_AP2-HKG-CCP-UCM_12_201905281539_35251
cdr_AP2-HKG-CCP-UCM_06_201905281539_297892
cmr_AP2-HKG-CCP-UCM_06_201905281539_297892
cdr_AP9-SYD-UCCE-UCM_05_201905281539_200435
cdr_AP2-HKG-CCP-UCM_11_201905281539_98483

However, I can't get the ListFile processor to work and find those files. Any ideas on debugging strategies?

[#2]

Check what user is the owner of nifi process and whether has privileges to read those files or not.

[#3]

I can see and read the files from inside the nifi container. Not sure why the nifi ListFile process isn't picking them up.

nifi@4f614c6c67c4:/data/sftp/cdr$ ls -1t | head -5 | egrep '^c.r'

cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183

cdr_AP4-SYD-CCP-UCM_05_201905281913_53152

cdr_AP2-HKG-CCP-UCM_13_201905281912_89066

cdr_AP9-SYD-UCCE-UCM_07_201905281911_196358

cdr_AP8-HKG-UCCE-UCM_02_201905281910_243229

nifi@4f614c6c67c4:/data/sftp/cdr$ tail -n 5 cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183

1,8,973062,138959080,1559070286,8,0,0,"","",0,0,4,0,0,0,0,0,0,0,0,0,0,"0","0",138959081,3,0,1054035722,"b003864986001","b003864986001","",0,0,4,0,0,0,0,0,0,0,

[#4]

I can find and read the files from inside the nifi docker container. Not sure why ListFile cannot:

nifi@4f614c6c67c4:/data/sftp/cdr$ ls -1t | head -5 | egrep '^c.r'

cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183

cdr_AP4-SYD-CCP-UCM_05_201905281913_53152

cdr_AP2-HKG-CCP-UCM_13_201905281912_89066

cdr_AP9-SYD-UCCE-UCM_07_201905281911_196358

cdr_AP8-HKG-UCCE-UCM_02_201905281910_243229

nifi@4f614c6c67c4:/data/sftp/cdr$ tail -n 5 cdr_AP9-SYD-UCCE-UCM_08_201905281913_183183

1,8,973062,138959080,1559070286,8,0,0,"","",0,0,4,0,0,0,0,0,0,0,0,0,0,"0","0",138959081,3,0,1054035722,"b003864986001","b003864986001","",0,0,4,0,0,0,0,0,0,