I'm a new junior member of a team using NiFi and have what is probably a basic question, but I've been unable to find an answer. I need to be able to set the number of files to be stored in a destination directory (100, for example) and have it automatically start a new directory for each 100 files, which will be differentiated by naming directories with a timestamp. Can anyone help? Thanks!
You will probably need to use some combination of Wait/Notify Processors to maintain a 0-100 count at which point a new directory is made. Outside of that logic, the putHDFS would need to always be operating with the folder name generated dynamically once 100 counter is satisifed.