About MattWho

MattWho · ‎07-31-2018

@Veerendra Nath Jasthi If the ConsumeKafka processor is hanging (showing active threads but not generating any FlowFiles) every time you use it, there is some issue going on in the connection. Getting a NiFi thread dump and inspecting that thread dump for consumekafka should lead you to a thread that is waiting on something. - It may be waiting on some authentication issue with the keytab/principal... It may be hung with "promptForName" keyboard interactive auth which NiFi can't do... - The above us typically an issue seen when the ticket expires and has not be renewed. - Thanks, Matt

MattWho · ‎07-30-2018

@Sri Bet When a node attempts to join an existing cluster there are 3 files that are checked to make sure they match between the node connecting and the cluster: 1. flow.xml.gz (Contains everything you have configured/added via the NiFi UI.) 2. users.xml (Contains all users added via the NiFI UI. This file will not exist if you are not using NiFi's default file based authorizer. Also will not exist if NiFi is not running secured) 3. authorizations.xml (Contains all authorization policies which have been granted to above users added via the NiFI UI. This file will not exist if you are not using NiFi's default file based authorizer. Also will not exist if NiFi is not running secured) - I am not sure why one of your nodes had a mismatch in your authorizations.xml file. Perhaps while this node was not in the cluster a new policy was granted to a user? - Deleting the users.xml and authorizations.xml files is not the best resolution. When these files do not exist, NiFi will create on the basic authorizations for the initial admin identity and nodes defined in the authorizers.xml file. deleting these files will result in loss of any other authorizations that ay have been granted later. - The best fix is to copy the users.xml and authorizations.xml from another node to the node having issues. - Thank you, Matt

MattWho · ‎07-25-2018

@Harish Vaibhav Kali - You can copy the users.xml from one Nifi to another new NiFi installation without needing to edit it all assuming target is a standalone NiFi install. - Things become more complicated when your new installation is a NiFi cluster since the users.xml file you are copying over will contain user identifiers for the nodes on the original cluster it was copied form and not the new cluster. This would require you to hand edit the users.xml file before you could use it on new cluster will proper "identity" for each Nifi server in new cluster. - While this will add all the users from this users.xml file, there will only be authorizations setup for the user who was configured as the "initial admin identity" in authorizers.xml (that user must exist in the users.xml you copied over) and authorizations for the nodes "Node Identity <num>" in authorizers.xml (nodes must exist in copied over users.xml prior to starting NiFi). - You could also use the NiFi rest-api to add user on a new NiFi installation using the initial admin credentials to do so via the /nifi-api/tenants/users rest-api endpoint. - The least painful method is using NiFi's ldap sync capability to sync user/groups from ldap. Then there is no need to copy over a users.xml. All users would just come from ldap and an admin could simply assign the user authorization polices as needed. - thanks, Matt

MattWho · ‎07-25-2018

@Harish Vaibhav Kali - The users.xml only contains all the users and the UUID each has been assigned. It has nothing to do with authorizations for those users. While you can copy the users.xml file to another NiFi install which will result in those 10 users existing in that new NiFi, the authorizations for those 10 users will not exist on new server since the actual authorizations are stored in the authorizations.xml file. - Thanks, Matt

MattWho · ‎07-25-2018

@Frank Gaibler - You have configured your RPG to connect to a target NiFi instance/cluster via https://172.18.61.254:12443/nifi. During the two-way TLS handshake the target NiFi server presents its server certificate which does not contain "172.118.61.254" as its CN or as a SAN entry in the certificate. So it appears as though some other server is trying to pretend to be intended destination. This cause the handshake to fail. - You can add that ip to the server cert as a SAN. But you also must keep in mind that this initial communication performed by the RPG is only to retrieve S2S details from the target NiFi instance. The "details' sent in the response will include the hostnames for each target NiFi instance/node as configured in "nifi.remote.input.host" property set in the nifi.properties file. When it comes to actual data transfer between source Nifi (one with RPG) and destination NiFi it will be sent to this returned hostname(s). So you may find yourself needing to set that property in the target NiFi's nifi.properties file to the static IP for each target instance/node set in your firewall. - Thanks, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer.

MattWho · ‎07-25-2018

@Mohammad Soori - *** Forum tip: Please try to avoid responding to an Answer by starting a new answer. Instead use the "add comment" tp respond to en existing answer. There is no guaranteed order to different answers which can make following a response thread difficult especially when multiple people are trying to assist you. - Based on what you are showing me, your flow is working as designed. Since you have added all three outgoing relationships to the outgoing connection of the splitText processor, you would end up with duplication. - the "original" relationship is basically a passthrough for the incoming Flowfiles to splitText. This relationship is often auto-terminated unless you need to keep the original un-split flowfiles for something else in your flow. IN that case the original relationship would be routed within its own outbound connection and not in the same connection as "splits". - The fact that splitText is not really splitting your source Flowfiles (4 in and 4 out) tells me that the 4 source Flowfiles created do not contain any line returns from which to split that text. So the question is what does the output of one of these ~115 byte FlowFiles look like? - I also do not recommend routing the "failure" relationship along with "success" or "original" in the same connection. Should a failure occur, how would you easily separate what failed and what was successful. - Thank you, Matt

MattWho · ‎07-24-2018

@Hemantha kumara There are no rest-api hooks in to the nifi configuration files like the authorizers.xml. If you are asking how to use the rest-api to add new users (users.xml <-- done via "tenants" rst-api endpoints) and/or authorizations (authorizations.xml <-- done via the Policies rest-api endpoints), that is a different question. - I suggest starting a new question in the forum. - Thanks, Matt

MattWho · ‎07-24-2018

@Harish Vaibhav Kali Are we still talking about users created in a NiFi 0.x version being used in a NiFi 1.x+ version? How user authorization is handled is different between each of those major release versions. - Lets assume you are working with purely NiFi 1.x+ versions. There are three tightly couple files when it comes to user authorization (assuming you are using NiFi default file based authorizer): 1. users.xml <-- contains all user to which authorizations will be granted. Each user is assigned a unique UUID. 2. Authorizations.xml <-- Contains all authorization policies (some policies will be based off specific components uuids) User UUIDs are associated to one or more policies. 3. flow.xml.gz <-- contains all components add to NiFi and their configurations - So saving off these three files before your re-install your NiFi instance will allow you to put them back afterwards. Keep in mind that sensitive properties (i.e - passwords) in the flow.xml.gz file are encrypted using the sensitive props key set in the nifi.properties file. So you must use same sensitive props key on re-installed NiFi version or NiFi will fail to start because it cannot decrypt the sensitive props. - Thanks Matt

MattWho · ‎07-23-2018

@Hemantha kumara - When a new node attempts to join an existing cluster, three files are compared between the cluster and teh node requesting to join that cluster: 1. flow.xml.gz 2. users.xml 3. authorizations.xml - In your case it is complaining about your authorizations not matching. Normally with a new node, if the above three files do not exist, the new node will inherit them from the cluster. Often times user copy all the config files from the cluster to the new node when standing it up. This can be an issue if you copied the authorizers.xml file from a cluster node that has initial admin identity and node identity values set. With these set, that node will generate a basic users.xml and authorizations.xml files from those values. Since you cluster probably has users.xml. and authorizations.xml files that have been updated over time they will not match. - So you have two options moving forward: 1. Copy the current users.xml and authorizations.xml files from anyone of the cluster nodes to this new node replacing the locally generated copies. 2. Delete the users.xml and authorizations.xml files on this new node. Edit the authorizers.xml and remove initial admin and node identity from that file. restart node so it instead inherits those files from the cluster. - IMPORTANT NOTE: All you nodes in your cluster are granted specific policies. The most important of which is the "Proxy user requests" policy. I recommend adding your new node to this policy before actually joining it to the cluster. - Thank you, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer.

MattWho · ‎07-23-2018

@Alvin Jin There is no way to disable the strict hostname checking. The purpose is prevent NiFI from responding to requests that were directed at a different target host. - If the incoming request has hostname abc.example.com in the header, It is expected that the PrivateKeyEntry in the keystore being used by NiFi has that exact same hostname defined either as its CN or as a Subject Alternative Name (SAN). - NiFi does provide a mechanism to get around this hostname checking in Apache NIFi 1.6.0 or (HDF 3.1.x) versions. - This new property would get added to the nifi.properties file. - Thank you, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer.

Online	Offline
Last Visited	‎12-22-2025 07:04 AM

Member Since	‎07-30-2019 10:41 AM
Last Visited	‎12-22-2025 07:04 AM
Posts	3,406
Kudos received	1618

Cloudera Community

Re: Error importing NiFi workflow template from ve...

Re: Error importing NiFi workflow template from ve...

Re: How to elevate a default nifi user to admin - ...

Re: NiFi EnvokeHTTP - putting current date on HTTP...

Re: Invoking Nifi rest api in Data Flow

Re: In a Nifi-workflow some of the processors are ...

Re: nifi : Failed to connect node to cluster becau...

Re: NiFi v0.6.1 vs NiFi v1.5.0 - Provide access to...

Re: NiFi v0.6.1 vs NiFi v1.5.0 - Provide access to...

Re: Secure Remote Process Group communication with...

Re: Nifi TailFile Processor does not detect vast i...

Re: Unable to join Nifi cluster(tls is enabled) dy...

Re: NiFi v0.6.1 vs NiFi v1.5.0 - Provide access to...

Re: Unable to join Nifi cluster(tls is enabled) dy...

Re: NiFi 1.5 System Error: Invalid host header