@Bern  Is your external Zookeeper installed on the same host as your NiFi?   If so, the load your NiFi is putting on those nodes may contribute to the performance of your ZK.  The last Apache NiFi 1.x major release version if NiFi 1.28.  I recommend upgrading to this version.  You'll potentially need to make significant changes and updated to your Apache NiFi 1.x versions dataflows before they can be used in Apache NiFi 2.x.   Apache NiFi 1.28 is also new enough that it will produce the flow.json.gz format that is also used by Apache NiFi 2.x.   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Bern  The single-user-provider authentication login provider and Single-User-Authorizer are extremely basic and only intended for out-of-the-box Apache NiFi product evaluation.  Apache NiFi also generates simple self-signed certificates to support the secured connection over HTTPS.   For more robust security you should be using a different multi-user authentication provider like ldap-provider and a multi-user authorizer like the managed-authorizer.  You should also generate signed certificates.  The Self-signed certificates generated by NiFi will eventually expire. Reference: User Authentication Multi-Tenant Authorization   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Matt, After some digging I found that I could use an updateattribute processor.  In it I create two custom attributes: since_date ${now():format("yyyy-MM-dd")} since_date_1 ${now():toNumber():minus(86400000):format("yyyy-MM-dd")} Then later in the InvokeHTTP processor I can use the custom attributes in the url line: ... &since=${since_date}&until=${until_date}&earliest=true Even later I then learned those attributes can be created in the GenerateFlowfile processor or used directly in the url as the formula: ${now():format("yyyy-MM-dd")} which is along the lines of what you posted.  Thank you for the response. ... View more

@Akram-Khalil  What do you see logged in the nifi-app.log when you attempt to access the NiFI UI? I don't think this is related to your ldap configuration, but I don't have your authorizers.xml or nifi.properties to verify your configuration setup. This exception is more related to authorization and not authentication.  It is  more likely related to missing "proxy user requests" authorization being granted to the NiFi node certificates. But this should be easy to resolve if enough information can be shared, which includes nifi-userlog output which will show the user client identity being denied authorization and the above mentioned configuration files. You can also file a Cloudera support ticket if you have a Cloudera support contract and this can be solved live over a call.  Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Thanks for the replies! I ended up using the "https://${hostname(true)}:8443/nifi-api" endpoint in the InvokeHTTP processor. However, the DefaultSSLContext controller service that Data Flow comes out of the box with did not work due to trust issues.  ... View more

@Sanga  What reason is being logged by the elected cluster coordinator node when node-0 gets initially disconnected?  Is it a lack of heartbeat? I understand that onc it gets initially disconnected, NiFi cluster coordinator requests that it reconnect on next heartbeat and the triggered flow synchronization result in a exception blocking the node from rejoining the cluster. My initial thought is  that you may be hitting NIFI-12969 or NIFI-12232 which was addressed in Apache NiFi 1.26.  An upgrade may help with your issue.   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Rohit1997jio  I would discourage you from continuing to any automation using Apache NiFi 1.x "templates".  NiFi templates were deprecated in the now end of life Apache NiFi 1.x major release version.  They were officially removed in the Apache NiFi 2.x major release. versions. Uploaded NiFi templates consume valuable NiFi heap memory.  The replacement is NiFi Flow Definitions.   Apache NiFi started moving away from XML based configuration files in around Apache NiFi 1.16.   While all NiFi 1.x retained the flow.xml.gz until its final release version, the flow.json.gz was introduced in 1.16 and is what is loaded on NiFi startup.   The only difference is the deprecated flow templates allowed you to create a template that consists of just individual low level components, while flow definitions can be created at only the "Process Group" level. Flow Definitions can be created by downloaded the flow definition directly from a process group.  MattWho_0-1764689673153.png   Deploying Flow Definitions is also easier.  Simply drag the add new "Process group" icon to canvas and click on the "Browse" icon in the Process Group Name field box.  You will be prompted to provide the downloaded flow definition json file. MattWho_1-1764689858845.png This upload can also be done via a rest-api call. The Apache NiFi 1.x major release REST API covers the api calls to do everything that you can accomplish via directly in the NiFI UI. But you can also use your browser build in "Developer Tools" capture these rest-api calls as you perform the actions directly in the NiFi UI.   The Developer tools even give you an option to "Copy as curl" making it even easier to learn (browser will add many unnecessary headers you can opt to remove.  If you remove a header that is needed, the curl command will fail):  MattWho_2-1764690569584.png   NOTE: Apache NiFi 1.x is end of life and no longer being contributed to.  This means no more bug and CVE issues being addressed in the the 1,x release line. The Apache NiFi 2.x major release is the new supported major release (where templates no longer exist). Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt     ... View more