@Anishkumar Valsalam Not sure exactly what you are looking for. - Verify you can access the NIFi UI from any one of your nodes URLs. - Verify the UI shows 3/3 connected nodes. - Verify you can add components to the canvas (processors and connections) - Verify you can add controller services and reporting tasks. - Go into "cluster" UI and manually disconnect the "cluster coordinator" and/or " primary node". Make sure that a new node(s) is assigned these roles. - From "Cluster" UI make sure you can reconnect a disconnected node. Since your NiFi cluster is running unsecured there isn't much more to validate from an administration standpoint. - You don't have node to node mutual authentication to worry about - You don't have user authentication or authorization to worry about. Dataflows are built in a cluster the same way they are built in a standalone NiFi installation. Just keep in mind that every node in your cluster runs its own copy of the dataflow and works on its own set of FlowFiles. Thanks, Matt ... View more

@Ayaskant Das Based on the output from the nifi-user.log, the following user successfully authenticated to NiFi but was not authorized view the NiFi UI. Did you configure this user as your "Initial Admin Identity"? The "Initial Admin Identity" is configured in the authorizers.xml file and is used to initially create both the users.xml and authorizations.xml files. It is case sensitive and white spaces do count. If you NiFi had previously been started with an incorrect value for "Initial Admin Identity" you will need to correct the authorizers.xml with the correct value, delete both users.xml and authorizations.xml files, and restart NiFi so that these files are created again. Thanks, Matt ... View more

@Ayaskant Das @Joe Petro NiFi must be secured to run over HTTPS using a server certificates (loaded into a keystore) and a truststore before you can enable some form of user authentication. You can create your own keystore and truststore using the below procedure: https://community.hortonworks.com/articles/17293/how-to-create-user-generated-keys-for-securing-nif.html Note: If you are going to be using LDAP or Kerberos for user authentication, you can skip the last part of the above procedure about generating a user SSL certificate to load in your browser. Thanks, Matt ... View more

@Eric Lloyd Did you try using a "Demarcator" between your merged content? The MergeContent processor allows you to designate a Demarcator. Perhaps use new line or something else unique that can then be used later to differentiate between bits of content in the same HDFS file. Thanks, Matt ... View more

@Eric Lloyd The MergeContent processor adds FlowFiles from the incoming queue to virtual bins. Once the configured criteria on a bin is met all the FlowFile in that Bin are merged. So if you want to continue to merge incoming FlowFiles until X amount of time has passed then setting the "Max bin age" property is what you want. Note: Be careful how many FlowFiles you merge. The FlowFile attributes for all incoming FlowFiles being merged in a single bin live in the NiFi JVM heap memory. Merging to many FlowFiles at once can result in OutOfMemory (OOM) errors. There is no formula for the exact number you can merge per bundle/bin. It depends on how many attributes exist on a FlowFile and how large the values are associated to those attributes. Thanks, Matt ... View more

@Ayaskant Das NiFi by default will use a user's SSL certificate if it is included by your browser during the connection to NiFi's URL. NiFi can be configured to use LDAP or Kerberos as alternate Authentication methods. Once configured, these alternate methods will be used only if a user does not pass a SSL certificate. Information about setting up LDAP or kerberos can be found in NiFo's Admin guide: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user-authentication Thanks, Matt ... View more

@Mark Heydenrych While I like the idea, there is currently no way to have a log message written to a FlowFiles attribute upon routing to a failure relationship. You may want to open an Apache NiFi Jira around this idea. Typically the "failure" relationship is routed back on the source processor so that multiple attempts can be made to deliver the file. In cases like network hicups, duplicate files, etc. this makes a lot of sense. When dealing with processor config failures, permissions issues, etc. the file will never be successful. You could set up a failure count loop. This loop would create an attribute on FlowFiles that are routed to "failure" and continue to loop them back on PutHDFS until the count has reached a configured number. Once that count is reached, the FlowFiles could be routed out of the loop. You could then send a notification via putEmail of the failed FlowFile for user investigation. Here is a link to a retry count loop flow NiFi template: https://cwiki.apache.org/confluence/download/attachments/57904847/Retry_Count_Loop.xml?version=1&modificationDate=1433271239000&api=v2 Thanks, Matt ... View more