Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Multiple flowfile filename attribute comparison across filesystem folders - Process only unprocessed files

Highlighted

Multiple flowfile filename attribute comparison across filesystem folders - Process only unprocessed files

New Contributor

Hi,

I have a scenario where I have two folders present in the filesystem. The first is SrcFiles and the second is TgtFiles. Each file present in those two folders has a filename format as 'abc_YYYYMMDD.csv'. Now, I would only like to process those files from the SrcFiles folder which are not present inside the TgtFiles folder. It pertains to the concept of only processing those files for which the target file marker is not present. So far, i have created two separate input streams in the process group and could do the substring to extract the date to do the comparison but then how should i compare those two separate streams/file names in the src folder with the file names in the tgt folder? any help is appreciated.

Thanks!

Regards,

Umair

Don't have an account?
Coming from Hortonworks? Activate your account here