@Harish Vaibhav Kali - This thread is kinda moving off topic form the original question which has been answered. It is probably best to start a new question. - That being said, what you are showing me looks to be correct functionality provided the following is true: ---> When NiFi was started for the first time there was no pre-existing flow.xml.gz. For a brand new secure flow, providing the "Initial Admin Identity" gives that user access to get into the UI and to manage users, groups and policies. But if that user wants to start modifying the flow, they need to grant themselves policies for the root process group. The system is unable to do this automatically because in a new flow the UUID of the root process group is not permanent until the flow.xml.gz is generated. If the NiFi instance is an upgrade from an existing flow.xml.gz or a 1.x instance going from unsecure to secure, then the "Initial Admin Identity" user is automatically given the privileges to modify the flow. - Also keep in mind that if the users.xml and authorizations.xml files do not exist and you have configured both the "initial admin Identity" and provided a legacy "authorized-users.xml" file, Nifi will fail to start. That is because the initial seeding of the users.xml and authorizations.xml files can be done via one or the other, but not both. - Thank you, Matt ... View more

Thanks for your input. As my question was related to the previous ones,I put it there, but I think you're right. I started a new one. https://community.hortonworks.com/questions/210117/elasticsearch-dont-show-all-of-the-snort-s-index-1.html ... View more

make sure that you install only Nifi on these nodes. Do not club Kafka and zookeeper on nifi nodes. This will imporve network i/o performance. Dhieru ... View more

Thank you for your answer, The second hash is performed after a PutHDFS and not a ListHDFS (I have edited my post, sorry for the mistake). If I understand you well, this check is not useful because PutHDFS and FetchFile processors are already able to catch corruption errors reliably? I would compare both hashes with NiFi expression language (:equals function) inside a RouteOnAttribute processor. ... View more

@umang s Based on your flow design above, it looks like you are trying to route FlowFiles by comparing attribute between two different FlowFiles? That will not work. NiFi is looking for both ${temp_array} and ${category} to exist on same flowfile being evaluated by the RouteOnAttribute processor. ... View more

@Derek Calderon - Sorry to hear that. I did share this HCC link with a few devs I know if they have time to assist. - Thanks, Matt ... View more

@Derek Calderon - Short answer is no. The ExecuteSQL processor is written to write the output to the FlowFile's content. - There is an alternative solution. You have some processor currently feeding FlowFiles to your ExecuteSQL processor via a connection. My suggestion would be to feed that same connection to two different paths. The first connection feeds to a "MergeContent" processor via a funnel and the second feeds to your "ExecuteSQL" processor. The ExecuteSQL processor performs the query and retrieves the data you are looking for writing it to the content of the FlowFile. You then use a processor like "ExtractText" to extract that FlowFIles new content to FlowFile Attributes. Finally you use a processor like "ModifyBytes" to remove all content of this FlowFile. Finally you feed this processor to the same funnel as the other path. The MergeContent processor could then merge these two flowfiles using the "Correlation Attribute Name" property (assuming "filename" is unique, that could be used), min/max entries set to 2, and "Attribute Strategy" set to "Keep All Unique Attributes". The result should be what you are looking for. - Flow would look something like following: Having multiple identical connections does not trigger NiFi to write the 200 mb of content twice to the the content repository. a new FlowFile is created but it points to the sam content claim. New content is only generated when the executeSQL is run against one of the FlowFiles. So this flow does not produce any additional write load on the content repo other then when the executeSQL writes its output which i am assuming is relatively small? - Thank you, Matt ... View more

Ok, thanks Matt, this template helps me ... View more

perfect, thanks guys! ... View more