@Akram-Khalil  What do you see logged in the nifi-app.log when you attempt to access the NiFI UI? I don't think this is related to your ldap configuration, but I don't have your authorizers.xml or nifi.properties to verify your configuration setup. This exception is more related to authorization and not authentication.  It is  more likely related to missing "proxy user requests" authorization being granted to the NiFi node certificates. But this should be easy to resolve if enough information can be shared, which includes nifi-userlog output which will show the user client identity being denied authorization and the above mentioned configuration files. You can also file a Cloudera support ticket if you have a Cloudera support contract and this can be solved live over a call.  Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Sanga  What reason is being logged by the elected cluster coordinator node when node-0 gets initially disconnected?  Is it a lack of heartbeat? I understand that onc it gets initially disconnected, NiFi cluster coordinator requests that it reconnect on next heartbeat and the triggered flow synchronization result in a exception blocking the node from rejoining the cluster. My initial thought is  that you may be hitting NIFI-12969 or NIFI-12232 which was addressed in Apache NiFi 1.26.  An upgrade may help with your issue.   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Rohit1997jio  I would discourage you from continuing to any automation using Apache NiFi 1.x "templates".  NiFi templates were deprecated in the now end of life Apache NiFi 1.x major release version.  They were officially removed in the Apache NiFi 2.x major release. versions. Uploaded NiFi templates consume valuable NiFi heap memory.  The replacement is NiFi Flow Definitions.   Apache NiFi started moving away from XML based configuration files in around Apache NiFi 1.16.   While all NiFi 1.x retained the flow.xml.gz until its final release version, the flow.json.gz was introduced in 1.16 and is what is loaded on NiFi startup.   The only difference is the deprecated flow templates allowed you to create a template that consists of just individual low level components, while flow definitions can be created at only the "Process Group" level. Flow Definitions can be created by downloaded the flow definition directly from a process group.    Deploying Flow Definitions is also easier.  Simply drag the add new "Process group" icon to canvas and click on the "Browse" icon in the Process Group Name field box.  You will be prompted to provide the downloaded flow definition json file. This upload can also be done via a rest-api call. The Apache NiFi 1.x major release REST API covers the api calls to do everything that you can accomplish via directly in the NiFI UI. But you can also use your browser build in "Developer Tools" capture these rest-api calls as you perform the actions directly in the NiFi UI.   The Developer tools even give you an option to "Copy as curl" making it even easier to learn (browser will add many unnecessary headers you can opt to remove.  If you remove a header that is needed, the curl command will fail):    NOTE: Apache NiFi 1.x is end of life and no longer being contributed to.  This means no more bug and CVE issues being addressed in the the 1,x release line. The Apache NiFi 2.x major release is the new supported major release (where templates no longer exist). Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt     ... View more

@zzzz77  When you executed your curl commands, did you check both the nifi-app.log and nifi-user.log for any ERROR or WARN log output that would provide more detail on the exception. The Apache NIFi 2.x Provenance rest-api endpoint request should look more like this: curl 'https://<HOSTNAME>:<PORT>/nifi-api/provenance' \ -H 'accept: application/json, text/plain, */*' \ -H 'content-type: application/json' \ -H 'Authorization:Bearer <TOKEN>' \ --data-raw '{"provenance":{"request":{"incrementalResults":false,"maxResults":1000,"summarize":true}}}' \ --insecure Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@hckorkmaz01  While you are currently still using Apache NiFi 1.x major release version, it has reached end of life and is no longer receiving contributions.  As such components will not get library updates or security fixes going forward.   Apache NiFi 2.x is currently active major release being contributed to in the community.  The PrometheusReportingTask was deprecated in Apache NiFi 1.x and officially removed in Apache NiFi 2.x major release.   So I would avoid using it as you will eventually need to move to Apache NiFi 2.x to maintain a secure supported product release.  But technically, this reporting task, while not well maintained in the community, is capable of creating a prometheus endpoint which exposes metrics for all components (includes connections) for consumption. That being said, Cloudera has taken steps to create Cloudera versions of many of the deprecated and removed components in Apache NiFi 2.x; as well as, introduced many components not available at all in any Apache release version (PrometheusReportingTask is not one of them that was retained). https://docs.cloudera.com/cfm/4.11.0/nifi-components-cfm/components/ NOTE:  You are already using a considerably older Apache NiFI 1.18 release.  Many bug fixes and CVEs security issues have been addressed since that release.  If you cannot yet move to Apache NiFi 2.x, you should at least be on the most recent release of Apache NIFi 1.28. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@zzzz77  To ask a community question: 1. Go to https://community.cloudera.com/ 2. Make sure you are logged in 3. Click on the "Ask a Question" button: 4. Fill out all form: Note: Adding "labels" make your question more visible to those community members that specialize in specific products. Hope this helps, Matt       ... View more

@hckorkmaz01  There are a few options here: SiteToSiteStatusReportingTask - Utilizes NiFi's Site-To-Site protocol to send Records to a NiFi Remote Input Port.  It can be configured to send specific component type metrics (Connection) in your case.   The Site-To-Site reporting tasks function as task specific remote process groups for generating and sending FlowFiles to the configured target Remote Input port. InvokeHTTP processor - Can be used to invoke the rest-api endpoint to get the metrics you want directly in a NiFi dataflow. Site-To-Site reference documentation: Admin guide: Site-To-Site Properties (core setup) User guide: Site-To-Site (user guide)   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Soli  Unfortunately, there is not enough shared to say what is taking so long during your startup.  Above log snippet shows ~.001 millisecond between each log line.  At 34,000 connections, that would be only 34 seconds.  Assuming About the same for other components, I don't think the synchronizing of components is what is taking the bulk of the 10-15 minutes of startup time.  In the nifi-app.log, what time is reported application start: INFO [main] org.apache.nifi.runtime.Application Started Application in <xxxxx> seconds What does each node report here?  Are all nodes taking roughly same time to start or is there any specific node taking longer then the other two? Is performance good once NiFi is up and running? Are all ~21,000 components in the running state? Any WARN or ERROR logging during startup? How many FlowFiles are queued per node? After NiFi is started, maybe collect a verbose NiFi diagnostics output for review. ./bin/nifi.sh diagnostics --verbose diag.txt You may also collect thread dumps  every minute during startup to see what thread during is taking so long. Hopefully some of this will help you see what is slowing your startup. There have been some improvements to startup time in Apache NiFi 2.x release. There were a few improvements to startup in Nifi 1.x releases, but those were made prior to 1.22.   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more