I believe the process you have is spot on and keeps the number of processors to a minimum. Matt ... View more

@Saikrishna Tarapareddy S2S does not use LDAP for server authentication. S2S uses the keystore and truststore provided in the nifi.properties file to establish a secured mutual authenticated connection between two secured NiFi instances/clusters. The destination NiFi dictate whether the S2S connection will be secure or not. If you have secured your destination NiFi, then only a source NiFi (one with the RPG) that has been configured with its own server keystore and truststore will be able to connect. Since S2S relies on certificates for mutual authentication. The user authentication you choose to use can be different on each NiFi installation. LDAP on one, certs on another, etc... Thanks, Matt ... View more

Also recommend against putting the quotes around your folder names ('MS1' should be just MS1). ... View more

@Raj B The toDate NiFi expression Language function expected the input to this function to define the current format of the value it is being passed. The result is the number of milliseconds since Jan. 1st 1970. The Format function will take a standard date of format number milliseconds since Jan 1st 1970 and convert it into the desired output format as defined in teh function. Assuming you have an attribute Abc.DateTimeOfMessage with a value of 20161011075959, teh following NiFi EL statement will produce the output '2016/10/11 07:59:59': ${Abc.DateTimeOfMessage:toDate('yyyyMMddHHmmss'):format('yyyy/MM/dd HH:mm:ss')} The above EL statement firs convert the dat you have into the standard date format (milliseconds since 1/1/1970) using the toDate function and then pass that result to the format function which converts a standard dat format into the desired output string you are looking for. *** An alternative EL that will yield the sam result is: ${Abc.DateTimeOfMessage:replaceAll('^([0-9]{4})([0-9]{2})([0-9]{2})([0-9]{2})([0-9]{2})([0-9]{2})','$1/$2/$3 $4:$5:$6')} The above uses the EL replaceAll function uses java capture groups to break apart the incoming function and then uses the values of those 6 capture groups to reconstruct the output in the format you want. There are even more ways, but I figured this is good enough. Thanks, Matt ... View more

@vlundberg This has nothing to do with being installed via Ambari. If the core-site.xml file that is being used by the HDFS processor in NiFi reference a Class which NiFi does not include, you will get a NoClassDef found error. Adding new Class to NiFi's HDFS NAR bundle may be a possibility, but as I am not a developer i can't speak to that. You can always file an Apache Jira against NiFi for this change. https://issues.apache.org/jira/secure/Dashboard.jspa Thanks, Matt ... View more

@Obaid Salikeen Unfortunately the answer is no at this time. NiFi has zookeeper as a dependency in HDF Ambari, so it is installed when the NiFi service is selected. Once NiFi is deployed there is nothing stopping you from updating the NiFi configs via Ambari to point at your existing Zookeeper you already have installed elsewhere. Keep in mind that other service within the HDF Ambari stack also rely on Zookeeper, so you may need to reconfigure them as well. Matt ... View more