@Saikrishna Tarapareddy I understand you concern. There is no notion of a template when looking at the components loaded on your canvas. A Template simply allows you to instantiate a collection of components in preconfigured setup to the canvas. Once those components are instantiated, they are treated no different then any other component you added manually. The underlying process is no different. While this is a valid use case, simply changing how this process works could have serious issues for those who do not expect it to work this way. This use case requires an entirely new process. There is work ongoing in the NiFi community to develop versioned templates (right now there is no notion of a version on templates you create). This new feature would allow you to create version controlled variations of a template. As part of this the instantiated components would be tied back to a specific template version. I do not know all the details of where this is going or when it will become reality, but should make it easier to swap out versions of the same template instantiated to your canvas. Thank you, Matt ... View more

@Saikrishna Tarapareddy A template is nothing more then a collection of NiFi processors, connections and controller services that includes all their unique configurations (except sensitive properties). There is no direct correlation between a template and any processors on the canvas. When you use a template to instantiate the components contained within it to the canvas, there is no back reference to the template. You are free to modify, delete, etc any component instantiated from that template. In addition, each component when instantiated gets its own unique UUID assigned to it. A good analogy would be a Word document. If you create a Word document named "mydoc1.docx" and then later decide to select "Save as." and save a copy as "mydoc2.docx". Going back to mydoc1 and making changes will not apply to mydoc2. NiFi templates behave similarly. Once NiFi has successfully instantiated a template to the canvas it never goes back and looks at that template again to do any sort of comparison. It would be dangerous for NiFi to assume what changes should and should not be applied if it did such a comparison. Would you want it to alway blank sensitive properties. It is very common that adjustments are made to components instantiated form a templaet post being created. So the same template instantiated multiple times within the same NiFi is very likely to diverge in configuration from each other over time. Thanks, Matt ... View more

@Hadoop User If merging FlowFiles and adding more concurrent tasks to your putHDFS processor help with your performance issue here, please take a moment to click "accept" on the above answer to close out this thread. Thank you, Matt ... View more

@Anishkumar Valsalam If the answer provided solved your issue, please take a moment to click "Accept" below the answer to close out this thread. This will assist other who are searching for a solution to the same problem. Thank you, Matt ... View more

@Dhamotharan P I highly recommend staring a new HCC question for your particular use case so it gets better visibility. You van always reference this HCC in your new question. The bulletin and any stack trace from the nifi-app.log when you take your end-point down would also be very helpful. Perhaps it is doing a session rollback instead which leaves the file on the inbound connection rather the routing to failure. Thank you, Matt ... View more

@Andre Labbe Another tip: use the @<username> in your responses/comments so that the user is notified when you add a response/comment. ... View more

@Andre Labbe Tip: try to avoid creating new answers for every correspondence. Add comments to existing answers to continue that discussion thread. 1. It is possible to have a one node cluster, but not very efficient. By setting it up as a cluster you are adding the overhead of zookeeper that is not needed with a standalone Nifi install. With regards to your "An unexpected error has occurred", this has nothing to do with there only being one node. Something else is going on in that would need to be investigated via the nifi logs (nifi-app.log, nifi-user.log, and/or nifi-bootstrap.log). 2. ExecuteScript->ExecuteStreamCommand->ExecuteScript-> ExecuteScript->InvokeHTTP. (That is a lot of custom code in a dataflow.) Do you really need so many script/command based processors to satisfy your use case? You also must remember that each node runs its own copy of the dataflow and with the exception of the cluster coordinator, has no notion of the existence of other nodes. A node simply run the dataflow independently and works with it sown set of files and NiFi repos. So make sure that your scripts exist on every node in your cluster in the same directory locations with proper permissions. There is nothing fancy about zookeeper needed by NiFi. Zookeeper is just another Apache project and has its own documentation ( https://zookeeper.apache.org/ ). You do not need a zookeeper node for every NiFi node. A 3 node ZK is perfect for most NiFi clusters. If you found this answer addressed teh question asked in this forum, please click "accept" . Thanks, Matt ... View more