@MukaAddA  Sorry, writing such script is not a strong area for me.  I just happened to notice you were doing a session.create instead of a session.get.  You may get better help by raising a new question on how to create a script to be executed by the ExecuteScript processor to accomplish your use case and provide details on that use case.  I am sure there are others in the community that are good at writing such scripts. Matt ... View more

Hi everyone,  I experienced the same error. After inspecting also the logs of nifi-registry, I found the error  2023-09-04 16:18:10,346 ERROR [NiFi Registry Web Server-17] o.a.n.r.web.mapper.ThrowableMapper An unexpected error has occurred: org.apache.nifi.registry.flow.FlowPersistenceException: Git directory /data/nifi01/nifi-registry-1.18.0/../nifiregistry_git is not clean or has uncommitted changes, resolve those changes first to save flow contents.. Returning Internal Server Error response. org.apache.nifi.registry.flow.FlowPersistenceException: Git directory /data/nifi01/nifi-registry-1.18.0/../nifiregistry_git is not clean or has uncommitted changes, resolve those changes first to save flow contents.   I changed to the path noted in the error message, changed to the user, which executes nifi-registry and checked the git repository status: git status Several files were modified and the git directory clearly was not "clean". I just committed and pushed everything (I had to set the git user name to make a successful commit).  I don't know, why this all happend. But for now, its fixed.  ... View more

@mslnrd  This is likely caused by LDAP on 636 uses referrals that can your initial query can be referred to across the entire domain tree across multiple LDAP servers. So somewhere within that referral your issues arrises in the hostname verification. Switching to the global catalog port 3269 and there are no referrals.  I can't speak to the issues within your ldaps servers causing the issue within the referrals, but makes sense why switching to the secure global catalog port resolved your issue.   Hope this clarifies why the change in port resolved your issue. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@kothari  It is not Ranger's job to inform the client applications using Ranger what users belong to what group.  Each client application is responsible for determining which groups the user authenticated into that service belong to. The policies generated by Ranger are downloaded by the client applications.  Within that downloaded policy json will be a resource identifier(s), list if user identities authorized (read, write, and/or delete) , and list of group identities authorized (read, write, or delete) against each resource identifier.  So when client checks the downloaded policies from Ranger it is looking for the user identity being authorized and if client is aware of the group(s) that user belongs to, will also check authorization for that group identity.   so in your case, it i s most likely that your client service/application has not been configured with the same user and group association setup in your Ranger service.   If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

Thank you Matt it worked! ... View more

@sahil0915  What you are proposing would require you to ingest into NiFi all ~100 million records from DC2, hash that record, write all ~100 million hashes to a map cache like Redis or HBase (which you would also need to install somewhere) using DistributedMapCache processor, then ingest all 100 million records from DC1, hash those records and finally compare the hash of those 100 million record with the hashes you added to the Distributed map cache using DetectDuplicate.    Any records routed to non-duplicate would represent what is not in DC2.    Then you would have to flush your Distributed Map Cache and repeat process except this time writing the hashes from DC3 to the Distributed Map Cache. I suspect this is going to perform poorly.  You would have NiFi ingesting ~300 million records just to create hash for a one time comparison.       If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@abdullahvvs Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks. ... View more

@learner-loading were you able to resolve your issue? If any of the above posts were the solution please mark the appropriate, as it will make it easier for others to find the answer in the future.        ... View more