@Bhar  Can you share more detail?  Without it, I would only be making random guesses. What version of Apache NiFi are using? Is this a single instance of NiFi or a NiFi multi-node cluster? How is your MergeContent processor configured? Thank you, Matt ... View more

@melek6199  What you have is an authorization issue. When you access you multi-node NiFi  cluster, you are authorized only into the node in which you authenticated.   When you make a request like List Queue or Empty Queue, you are making a request from one node to all the other nodes to list or empty the connection queue.   This means that the nodes themselves need to be authorized to request other nodes to share back their queue list or empty their target node queues. All 4 of your NiFi nodes should already have been authorized for "proxy user requests", but in order to list or empty queues, your node will need these additional authorizations: "view the data"  - authorizes a node(s) to list the data from other nodes (user must also be authorized) "modify the data" - authorizes a node(s) to empty a connection queue on other nodes. You can see from the nfi-user.log output you shared the identity and policy missing to perform this action on the specific connection UUID: Node x.x.x.x:8443 is unable to fulfill this request due to: Unable to modify the data for Processor with ID d3a802c6-0196-1000-ffff-ffff90fdc7b8 You would have seen this same exception for all but one node when you made the request to empty the queue. Authorizations are inherited form parent Process groups unless explicitly set on the individual component directly.   So you don't need to authorize your nodes for "view the data" and "modify the data" on the connection "d3a802c6-0196-1000-ffff-ffff90fdc7b8" directly, but rather set these authorization instead on the parent process group.  Keep in mind that child process groups also inherit from parent process groups unless policy is explicitly set on that child process group..  Typically you would set these authorization policies on the root process group (top level).  You'll also notice when you are viewing policies on a component it will tell you if it is inheriting policies and if you choose to set explicit policies on that component it asks you if you want to copy the inherited policy before  modifying. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Does anyone has an update on my request? ... View more

@agriff  I did not know that you were using the Apache NiFi 2.x release.   The component list I provided is from the Apache NiFi 1.x release. NiFi 2.x switched from having numerous client version Kafka based processors to single Kafka based processors that now use a KafkaConnectionService controller service  component to define the kafka client version.  In Apache NiFi the only connection service included is for theKafka 3 Client.  The Kafka client 3 I understand to be backwards compatible to Kafka 2.6, but sounds like you are having success with using it for Kafka 2.5. Glad to hear you were able to resolve yoru underlying schema issue. Setting Bulletins level on a processor has absolutely nothing to do with log levels written to the nifi-app.log.  It only controls what level bulletins are created within the NiFi UI.  To change logging within the NiFi logs, you will need to modify the logback.xml configuration file found in the NiFi conf directory.  Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@nifier  I would not expect much difference between making the stop request via the NiFi UI or via a rest-api call.  Even when you make a request to stop components via the NiFi UI, the UI may quickly show the "stopped" icon on the component, but any active threads are not killed in that process.  In fact the processor is considered "stopping" until all its active threads complete however long that takes.  While still in the state of stopping, you can not modify those components.   A component is considered stopping if its "activeThreadCount" is not 0. when you are executing your rest-api script without the delay, what exception are you encountering? This one? unable to fulfill this request due to: Cannot start component with <component id> because it is currently stopping  Above means you have active threads. Perhaps you can build a wait loop around above response until the active threads complete.  Or you can capture that component id and execute a terminate threads command on it. ../nifi-api/processors/<component id>/threads -X DELETE Terminating threads will not cause data loss.  NiFi is not killing any threads in this process, only way to kill threads is via a NiFi restart.  Terminating threads on component just shifts the thread to dev null and unhooks it from the FlowFile(s) it is associated with in the inbound connection. When the processor is restarted, the FlowFile(s) will be reprocessed by the component.  Should the "terminated" thread complete execution its logging and output just goes to dev null and results are not written back to a FlowFile, but depending on processor it could end up in duplicate data on a destination system if the tread is sending data out of NiFi since NiFi will reprocess the FlowFile originally associated with that terminated thread next time processor is started. The other option is to get the status of components for the process group you stopped and parse the json for any "activeThreadCount" were count is not 0 and wait 1 sec and make request again and then repeat this loop until all are 0 before making your next rest-api call. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more

Thanks @MattWho  for your valuable inputs. This is very helpful for conducting the correct analysis on our end. I will check the possibility of customizing the processors that connect to the database.  ... View more

@sydney-  The SSL handshake error you're encountering is a common issue when connecting NiFi instances to NiFi Registry in secure environments it indicates that your NiFi instances cannot verify the SSL certificate presented by the NiFi Registry server. javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider. certpath.SunCertPathBuilder Exception: unable to find valid certification path to requested target Based on your description, there are several areas to address. The certificate used by NiFi Registry is self-signed or not issued by a trusted Certificate Authority (CA) The certificate chain is incomplete The truststore configuration is incorrect 1. Certificate Trust Configuration Verify Certificate Chain: # Check if certificate is in NiFi truststore (repeat for each instance) keytool -list -v -keystore /path/to/nifi/truststore.jks -storepass [password] # Check if certificate is in Registry truststore keytool -list -v -keystore /path/to/registry/truststore.jks -storepass [password] # Verify the Registry's certificate chain openssl s_client -connect nifi-registry.example.com:443 -showcerts Ensure Complete Certificate Chain: Add the Registry's complete certificate chain (including intermediate CAs) to NiFi's truststore Add NiFi's complete certificate chain to the Registry's truststore # Add Registry certificate to NiFi truststore keytool -import -alias nifi-registry -file registry-cert.pem -keystore /path/to/nifi/conf/truststore.jks -storepass [password] # Add NiFi certificate to Registry truststore keytool -import -alias nifi-prod -file nifi-cert.pem -keystore /path/to/registry/conf/truststore.jks -storepass [password] 2. Proper Certificate Exchange Ensure you've exchanged certificates correctly export NiFi Registry's public certificate keytool -exportcert -alias nifi-registry -keystore /path/to/registry/keystore.jks -file registry.crt -storepass [password] Import this certificate into each NiFi instance's truststore keytool -importcert -alias nifi-registry -keystore /path/to/nifi/truststore.jks -file registry.crt -storepass [password] -noprompt 3. NiFi Registry Connection Configuration In your NiFi instance (nifi.properties), verify # Registry client properties nifi.registry.client.name=NiFi Registry nifi.registry.client.url=https://nifi-registry.example.com/nifi-registry nifi.registry.client.timeout.connect=30 secs nifi.registry.client.timeout.read=30 secs Verify these configuration files in NiFi (production/development) # nifi.properties: nifi.registry.client.ssl.protocol=TLS nifi.registry.client.truststore.path=/path/to/truststore.jks nifi.registry.client.truststore.password=[password] nifi.registry.client.truststore.type=JKS In NiFi Registry # nifi-registry.properties: nifi.registry.security.truststore.path=/path/to/truststore.jks nifi.registry.security.truststore.password=[password] nifi.registry.security.truststore.type=JKS 4. LDAP Configuration For your LDAP integration issues in authorizers.xml ensure you have <accessPolicyProvider> <identifier>file-access-policy-provider</identifier> <class>org.apache.nifi.registry.security.authorization.FileAccessPolicyProvider</class> <property name="User Group Provider">ldap-user-group-provider</property> <property name="Authorizations File">./conf/authorizations.xml</property> <property name="Initial Admin Identity">cn=admin-user,ou=users,dc=example,dc=com</property> <property name="NiFi Identity 1">cn=dev-nifi,ou=servers,dc=example,dc=com</property> </accessPolicyProvider> In the authorizations.xml add appropriate policies for the dev-nifi identity <policy identifier="some-uuid" resource="/buckets" action="READ"> <user identifier="dev-nifi-uuid"/> </policy> 5. Proxy Configuration For proxy user requests, add in nifi.properties nifi.registry.client.proxy.identity=cn=dev-nifi,ou=servers,dc=example,dc=com 6. Restart Order After making changes restart the Nifi instance in the below order NiFi Registry first Then restart all NiFi instances Happy hadoping ... View more

One more update if someone will read it in future. I sad that: I did it and restart NiFi service, but users.xml file was not updated. So, should I update users.xml manually? But I was wrong and my users.xml is correctly updates after user changes in NiFi UI. So, the question in not had a reason, all works fine. ... View more

@MattWho yeah, thanks a lot! You are totally right. 😃  ... View more