I am performing incremental load using Generate Table Fetch processor in NiFi. I may loose state of my processor if NiFi crashes. I want to figure out a way to store processor state in database and retain it whenever required. I referred state management in NiFi but have a doubt. 1) How to select a particular component to fetch its state using groovy script? 2) I think, when we run Groovy script to fetch state of component, it will fetch state of ExecuteScript who is running the script. Or can we fetch state of any processor? Thanks, Udayan ... View more

Hi, I am working on scenario where I need to pull data from table in Sql Server and Store it to some place. Currently, I have designed below workflow (at high level): GenerateTableFetch --> Execute Sql --> ConvertRecord --> PutFile I am working on a NiFi cluster and observed that, each node on cluster is executing this workflow individually and keeping output files on its local storage. I am looking for a way where output of GenerateTableFetch i.e. flow files with queries will get distribute equally to let say 4 nodes. Each node will have unique sets of queries to execute by Execute Sql task. Here, for example, if I have 12 GB Data and 4 nodes, and GenerateTableFetch is generating query to pull 1 GB data in one flow file, then each node should share the work and individually pull 3 GB data. Can someone please help me to achieve this? Also, if each node will deal to pull specific set of data then, What will happen when that node will go down? Is there any way such that if some node is failed then its work items will be shared with other nodes in cluster? Can you guys help? @Matt Clarke @Bryan Bende ... View more