@P2as  Zookeeper needs to have more then 1 node in order to have quorum. A single ZK instance will not work.   The next question is how is client/user authentication and authorization setup? While i dod not see any untrusted proxy exception or SSL exceptions in what you shared, I wonder if you are encountering a mutualTLS issue between your two nodes resulting in your connection exception. When you try to access a NiFi node's URL, your request is replicated by the elected cluster coordinator to all nodes in the cluster and those nodes need to respond with what access the authenticated user is authorized for on each node.  It is this replication request that is failing. You may need to dig a bit deeper in to your logs and configurations making sure that the NiFi instances successfully bound to those ports and those ports are not being blocked. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@yagoaparecidoti  There are still multiple ways to move complete dataflows from one NiFi to Another NiFi. When you start a NiFi for the first time, NiFi create the root Process Group (PG).   This is the blank canvas that you see when you log in.  I would suggest creating a Child PG at the root PG level and build all you dataflows  within the new child PG. Method 1: Use NiFi-Registry:  You can setup a Single NiFi-Registry that all your NiFi deployments can use. Start Version control on your child PG.  This will write the PG flow definition to that NiFi-Registry.  Give your other NiFi instance the ability to read the NiFi-Registry Bucket and flow in which that flow definition is stored. On your other NiFi instances import that version controlled PG from the shared NiFi-Registry. Anytime a newer version of that Flow definition is pushed to NiFi-Registry, all your other NiFi instances using it will show a newer version as being available. Method 2: You can utilize the rest-api to download a flow definition of that child PG. You can utilize the rest-api on other instances of NiFi to instantiate that downloaded flow definition onto the root PG of those instances. As far as the NiFi REST-API goes, it is always easier to utilize the "developer tools" available within your browser to capture the rest-api calls as they are made when you manually perform the steps directly via the NiFi UI.  This allows you to copy the rest-api call as a curl command.  From that captured curl command you can see the rest-api endpoint used to perform the request along with format of any data or variables needed in the request. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Leo3103  It appeasr you are handling the conversion of your NiFi flow Definition for use in your MiNiFi incorrectly as per doscumetation: https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md You should be downloading your flow definition (json file) via the NiFi UI. Then you should be renaming that file "flow.json.raw" (no mention of compression here) and place it in the MiNiFi conf directory. Once you have your flow.json.raw file in the minifi/conf directory, launch that instance of MiNiFi and your dataflow begins automatically. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@yagoaparecidoti  I think there is still confusion here on what you are really doing. Templates are in XML format and not JSON.  There is not option to download a template in JSON format. Templates and Flow Definitions are two different things (templates are deprecated and fully removed as of Apache NiFi 2) In Apache NiFi 0.x and 1.x versions you can write click on the canvas or on a selection of components and select "create template" from context menu displayed or by clicking teh create template option from the operate panel This results in the creation of an XML template that you need to navigate to the template UI under the NiFi global menu in order to download the XML template. Flow Definitions are in Json format and can be created by right clicking on process group or the canvas of the process group and selecting "Download Flow Definition".  This prompts the creation of flow definition json you store outside of your NiFi.  A Flow i Json is uploaded to NiFi by dragging the Process Group icon to the canvas and clicking on the browse icon to select your json for upload to the UI. Flow definitions are NOT stored within your NiFi. I would advise strongly against using NIFi templates as they will become unusable in the newer versions of NiFi. Templates where deprecated for two reasons: 1. They are created and held in NiFi which means they consume NiFi heap memory space until they are deleted via the template UI. 2. They use XML format.  NiFI has moved away from xml based flows.  The flow.xml.gz is also deprecated with the flow.json.gz format. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Salmidin  Please create a new community question with the details of your issue. It seem unrelated to this thread issue.  The "Policies" missing from the global menu indicates your NiFi is either using Single User Authorizer (default out of box) or is not setup to be secure. NiFi needs to be configured with a production ready authorizer for the "Polices" and "Users" options to be visible in the NiFi Global menu. Fell free to ping me @MattWho in your new community question. Thanks, Matt ... View more

@NagendraKumar  The image you shared indicates that your PublishKafka processor is producing a bulletin.  What is the nature of the exception being reported in that bulletin? I also see what appears to be only one connection exiting your PublishKafka processors.  The publishKafka processor has multiple relationships. Are you auto-terminating the failure" relationship?  If so, i never recommend doing that. ----------- Now when it comes to monitoring queue sizes/thresholds, you could use the ControllerStatusReportingTask NiFi reporting task to output these stats for all connections to a dedicated logs (see Additional Details... for how to setup dedicated logs via NiFi's logback.xml).   You can then create a NiFi dataflow that tails the dedicated connection log and parses the ingested log entries for connections where thresholds exceed your 80% threshold and route those to a putEmail processor for yoru notification needs. (preferred) Another option here is to use the SiteToSiteStatusReportingTask to report specifically on NiFi connections and feed that data in to a dataflow that parses for threshold in excess of 80% and route those to a putEmail processor.  This method has less overhead as it does not write to a NiFi log, require tailing logs, can be setup to report on connections, and reports in a more structured format (see Additional Details... ). ---------- From your image I can also see your PublishKafka processor reporting 3 actively running threads.  You mentioned the processor becomes hung?  Have you analyzed a series of thread dumps to identify where it is getting hung?   NiFi also offers a way to monitor for long running tasks: Runtime Monitoring Properties You could use this in conjunction with the SiteToSiteBulletinReportingTask to construct a dataflow that could send an email alert when task are detected on processor that have been running in excess of the configured threshold.  This runtime monitoring does have an impact on your overall NiFi performance due to the overhead needed to run it.  So if you find it impacts your throughput performance negatively, you'll need to stop using it. __________ Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@wasabipeas @Adhitya  In the thrown exception it reports which node has the mismatched revision. Is that node the currently elected cluster coordinator? Have you tried on just that reported node: 1. Stopping NiFi 2. Remove or rename both the flow.xml.gz and flow.json gz files (only deleting one will not work) 3. Restart that NiFi node.  It will inherit the flow from the cluster coordinator when it joins. If this node was the elected cluster coordinator, when you shut it down another node will assume the cluster coordinator role. ---- Another option is to disconnect the node reporting the mismatch in revision.  Then from the same cluster UI used to disconnect that node, select to drop/delete it from cluster.  Your cluster will now report one less node.  See if you can then move the process group or if it reports another node with mismatched revision? NOTE: Deleting/Dropping a node from cluster using the Cluster UI does nothing to that node.  If you restart that node that was deleted/dropped it will rejoin the cluster again.   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more