@Anishkumar Valsalam Each Node in a NiFi cluster runs its own copy of the dataflow. Nodes do not automatically share data between them. I agree that the simple flow described by mliem above is great to test dataflow performance per node, it does not test comms between nodes. 1. Try setting up a dataflow that uses S2S as well to distribute data between your nodes. GenerateFlowFile ---> RPG inputport ---> CompressContent --> updateAttribute. The above will show how network between nodes behaves. While the dataflow is running, try removing and/or adding an additional node to your NiFi cluster. 2. If you are running securely, make sure your user can successfully login to the UI of every node in your cluster and successfully make a change on the canvas. 3. Manually disconnect the node that is marked as the cluster coordinator and/or primary node. Verify that a new Cluster coordinator and/or primary node is elected. 4. Configure a processor to run on "primary node only". Verify that the node assigned "Primary" node is the only node running that processor. Shutdown current node elected as primary node and make sure whichever node is elected as new primary node starts running that processor. Thanks, Matt ... View more

@Chaitanya Manthine It appears Ambari is complaining that the following property was not set in your NiFi configs: Thanks, Matt ... View more

@Bram Klinkenberg The first thing that seems out of place to me is that in the following two lines the user DN is "CN=admin": ./tls-toolkit.sh standalone -n 'nifi.domeinbram.nl' -C 'CN=admin' -o keys/ and <propertyname="Initial Admin Identity">CN=admin</property> However, your users.xml file shows a DN of "cn=admin": <useridentifier="1a0ab441-da40-30dd-b28c-c4a4c710d03c"identity="cn=admin"/> They must match exactly or it is treated as a different identity. If you tail the nifi-user.log while you you try to access the UI, you will see lines output for authentication and authorization. You will see in that log the exact DN being passed to the authorizer. Is it mixed case or all lower case? The DN shown in the nifi-user.log must match exactly with what is in the users.xml file NiFi only generates the users.xml and authorizations.xml files on first startup when NiFi is secured. Subsequent changes to the authorizers.xml file will not trigger any changes/updates to pre-existing users.xml and/or authorizations.xml files. In your case since you are just getting started and you have no other users yet to worry about, you can simply delete these files and restart. NiFi will re-create them since they do not exist based on the current settings in the authorizers.xml file. In your case, you could also just manually edit the users.xml file since it appears to be a very simple change. Thanks, Matt ... View more

@Bala Vignesh N V The latest documentation for Apache NiFi can be found here: https://nifi.apache.org/docs.html You will want look in the "Getting Started" section for installing on a Linux based platform. Thank you, Matt ... View more

@Gowthaman Jayachandran The NiFi master bulletin board does not constantly run in the background, Its simply shows a continuous stream of bulletins being produced by processor components on the canvas while it is open. The bulletins are nothing more then snippets of the WARN or ERROR log lines that are being written to the nifi-app.log. The processors that are actually producing bulletins often have a a failure relationship where such FlowFiles will be routed upon failure. This is where you want to handle your notifications. Perhaps setting up a failure loop that will attempt to processes the FlowFile x number of times before kicking it out of the loop. If a file gets kicked out of the loop, you could use and putEmail processor to notify you of the failure. See below for template of such a failure loop: https://cwiki.apache.org/confluence/download/attachments/57904847/Retry_Count_Loop.xml?version=1&modificationDate=1433271239000&api=v2 For processor components where there is no failure relationship, you could use a MonitorActivity processor in your dataflow. This processor can be configured with a threshold and if a FlowFile has not passed within that threshold, a FlowFile is generated that can also be send to a PutEmail processor for alerting purposes. It also will Generate a FlowFile when activity has been restored. Thanks, Matt ... View more

@mayki wogno Is the same directory the same size of everyone of your zookeeper nodes? If not you may be having an issue on only one of your znodes. You should be able to shutdown the zookeeper node and purge all those files. The pertain files will be re-written from the other znodes in the zookeeper cluster when it rejoins the zookeeper cluster. Zookeeper is storing information about who is your current cluster coordinator, primary node, and any cluster wide state various from various processor in your dataflows. I am assuming you are running the embedded zookeeper here. In that case the zookeeper.properties file should control the auto purge of the snapshots through the following properties: autopurge.purgeInterval=24 autopurge.snapRetainCount=30 The transaction logs should be handle via routine maintenance which you can find here: http://archive.cloudera.com/cdh4/cdh/4/zookeeper/zookeeperAdmin.html#sc_maintenance Thanks, Matt ... View more