Hi, I have a requirement to handle 10k files in Nifi in parallel. Whats the best way to handle this scenario? Say for example if i use GetFile Processor and point to a loading directory, will it take 10k files one by one or it will be handled in parallel. Is there any properties or way to set it up so that 10k files will be loaded in parallel. Its not necessarily GetFile. ListFile-FetchFile also will do. Another question: Does nifi creates a single JVM instance to handle all the flowfiles being generated, or will there be multiple JVM's created for different processes? ... View more

Am pretty new to Nifi. The soltion we have designed is using Nifi for data ingestion and oozie for scheduling. Using oozie, hive tables are loaded, merged and hive-to-hive schema copy is performed. 1) Is it possible/ recommended using Nifi to completely achieve it. 2) What are the areas that needs to be taken care while using this approach? 3) The flow after developed using Nifi has so many common processors (updateattribute,putmail etc). Is there any way to set common processors as a separate template and make use of it wherever necessary) Note: The data flow is scheduled to run once in a day. ... View more