@Serhii K Best recommendation I can make is to stop using the embedded ZK capability in NiFi and stand up an external ZK (on 3 different servers ideally) for your NiFi cluster to use. - The issues come from using embedded ZK 1. If NiFi is stopped or rebooted, ZK also is stopped or restart which can easily cause connection issues across your cluster once you lose ZK quorum. 2. NiFi can be very resource intensive (disk, memory, and CPU) depending on your particular dataflows and data volumes. This can become an issue when both the ZK and NiFi services are competing for the limited server resources. - Thanks, Matt ... View more

@sunile.manjee You must "Stop" NiFI CA before the "Delete" option is available. Once it has been deleted, I would confirm contents of your keystore and truststore are still correct in case Ambari executed the tls-toolkit and overwrote them. ... View more

@sunile.manjee - Just to add some clarity. The NiFi CA is a self-signed certificate authority (Meaning the NiFi CA's certificate is a root level CA and has not had its certificate signed by a higher authority). The NiFi TLS toolkit can be used to create client/server certificates that are then signed by this authority for use by NiFi. The truststore.jks file used by NiFi typically contains all the "trustedCertEntries" needed to authenticate any client certificates used to authenticate against NiFi. So this would include the trustedCertEntry for the NiFi CA. The keystore.jks file used by NiFi could contain the PrivateKeyEntry (which is the certificate that has been signed by the NiFi CA). - Within a browser there are certificates and a number of default and possibly user added Authorities. In order for your browser to trust any certificate signed by the NiFi CA, the public certificate for your NiFi CA must be added to your browsers trusted authorities. - For example, you can see I have done that in Firefox here: You will see "Apache-NiFi CA" has been added. Now this browser will trust any certificate that has been signed by that authority. - Thank you, Matt ... View more

@Rinki - 1. The key term here is "copy". My understanding is that the Files consumed from the FTP server will not be deleted. So on the first of every Month you will list the exact same files over again. (Going to assume these files are maybe updated over the course of the month?). ---- Your best bet would be to use a combination of listFtp and fetchFTP processors. The listFTP processor can be configured with a file filter regex so that only files with matching that filter are listed. The output from this processor will be a 0 byte FlowFile for each File listed. At this point the actual content of these flowfiles have not been retrieved yet; however, each 0 byte FlowFile has numerous bits of metadata about the target file. These attributes can be used to filter out additional listed files by time constraint (perhaps using RouteOnAttribute processor). The listFTP processor also records state. On execution next month only files with a last modified date newer then the previous recorded state will be listed. The FetchFTP processor would then be used to fetch the actual content of those listed FlowFiles. - 2. At this point you have all your FlowFiles (Each FlowFile consists of FlowFile content and FlowFile attributes). These attributes will include everything you need except file type at this point. Since NiFi is data agnostic, it does not at its core care about content type/format (specific processors that operate against the content will). You may consider trying to use the IdentifyMimeType processor to get the file types. Route the "success" relationship twice. Once down your database processing path and once down you local server path. On database path you want to take the attributes needed for your DB and replace the content with that information (examples: ReplaceText, AttributesToJson, etc...) and the put to your DB. On local server path you want to write original content to your local system directory (Example: putFile) - 3. As far as scheduling, you just need to configure your original ListFTP processor using the Cron Driven scheduling strategy and configure a Quartz cron so that the processor is only scheduled to run on the 1st of every month. - Thanks, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer. ... View more

@sunile.manjee Good catch. Thank you for sharing. ... View more

@sunile.manjee I updated my last response, let me know if this resolves your issue. ... View more

@sunile.manjee - I was playing around with PutS3Object and do not see same issue. Your configuration screen putS3Object processor configuration above shows values being set for both "Access Key' and 'Secret Key'. Only way I can reproduce what you see is if you checked the box "Set Empty String" for the 'Access Key' and 'Secret Key'. The empty string is being treaded like a null value and process validates that as being configured (that is why it shows "Sensitive value set"). - What you want to do is select field and hit your delete key to clear it out rather then checking the box. The "Access Key' and 'Secret Key' fields should show "No Value Set" instead. At that point your processor should validate correctly. - Thanks, Matt ... View more