Yes this really helped as the problem was a misconfiguration. The truststore and keystore in Nifi Registry were both pointing to the truststore. On correcting this all is working fine.   Thanks   John ... View more

Thanks on hdf 3.4.1. cleared the users from the auth.xml and users.xml. back to normal operations. Appreciate your solution. ... View more

@johannes_meixne    The only place you would use "{0}" would be in the ldap-provider used for user authentication.  The {0} gets replaced with the username entered in the login UI.  The LdapUserGroupProvider however is used by the nifi authorizer to assemble a list of users and group strings (along with the association/membership between those user and groups) which can be used to assign authorizations against.  The LdapUserGroupProvider executes on NiFi startup and then on a configured schedule to refresh the synced users and groups.  Since it takes no input you can not use "{0}" in the search filter. ... View more

@Alexandros    As soon as you retrieve the json from which you later extract the sensitive values, those sensitive values are available/readable by anyone who has access to view the content of a FlowFile.  Even if you restrict user access so they can not view the FlowFile content, once you extract those sensitive values to FlowFile attributes they become exposed further.  There is currently no methods within NiFi for encrypting FlowFile attributes.  Doing so would also require any downstream processor in which you would want use those encrypted attributes to be able to understand that it is a sensitive value and decrypt it for use.  Bottom line here, is that the capability you are looking for does not exist within NiFi right now. This sounds like a new development opportunity/contribution maybe.  Perhaps a new NiFi controller service that handles pulling the credentials from the AWS Secret Manager and obtaining the JWT token without writing anything to the FlowFIle's attributes or content.  Then any processor you would want to use this new CS in would need to be extended to support the new capability.  I am not a developer myself, but this sound like non-trivial work. @alopresto might have some thoughts to add here. Thanks, Matt ... View more

@i_love_burger    I am not sure what configuration step you need help with. What is not "working"? What errors are you seeing when processor tries to execute? NiFi configured method for user authentication has nothing to do with how processors authenticate with external servers/resources.  Your user authentication and subsequent authorization simply controls what you as a NiFi user can see and do within the NiFi UI.  NiFi components like processors you add via the NiFi UI are not running/executing as your user.  All components are executed by the NiFi JVM service user.  Any authentication required is configured via the component itself. If your endpoint httphandler running on port 8060 required mutual TLS authenctication via the TLS handshake, you will need to configure a SSLContextService that has both a keystore and a truststore. The keystore must contain a single "PrivateKeyEntry" that supports clientAuth and is capable of beig trusted by the target endpoint.  The Truststore used must contain the complete TLS trust chain for the target endpoint presented server certificate in the TLS hanshake. If only one-way TLS is required, all that is need is an SSLContextService with the above mentioned truststore. Using command: "openssl s_client -connect <servername>:<port> --showcerts"  is a good way to observe the server side handshake and obtain all the public certificates needed for a complete trusts chain in your truststore. Hope this helps, Matt ... View more

@LearnerAdmin    It is not clear to me what you are asking when you say "add NIFI CA in authorities".   Instructions on using the NiFi TLS toolkit can be found here: https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_toolkit   Using the Client/Server Tls Toolkit operational mode covered here: https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#client-server Will give you the ability to create a running NiFi CA authority "server" which will sign your NiFi node certificates created using the "client" mode.   Thanks, Matt ... View more

@Griggsy    I'd strongly encourage you to start a new question rather then asking for help on an existing question with an already accepted solution.  You'll get better traction and visibility that way. Matt ... View more

@SamCloud    Not really how the processor is designed to work.  It order for a FlowFile to get outed to the duplicate path the distributed map cache must already have a matching entry in it.  All i can think for you to try is setting up a second distributedMapCache server where you use detect Duplicate a second time.  Those FlowFile routed down the duplicate connection the first time are used to populate the second cache server.  Then all your FlowFiles routed to non duplicate path originally are checked against that second distributedMapCacheServer.  Only thing here is you hav a bit of a race condition since you need to make sure the FlowFiles on the duplicate path are processed before those on the non-duplicate path.  For that you might want to use maybe the wait and notify processors. Wait on non duplicate path and the notify on after the detectDuplicate on the duplicate path.  You can see where this gets very complicated.  You may need to develop something custom here.   Matt ... View more

@ppbirajdar    My next thought would be to just drop the cloned FlowFile right before the MergeContent since you intent is not to reassemble the exact same original file.  Your intent, if i understand correctly, is to just create one file that contains 1 FlowFile with each unique fragment index.   If that is the case, perhaps all you need is a detectDuplicate in front of your mergeContent that uses a combination of Fragment.identifier and fragment.index for determining if there is a duplicate FlowFile to drop.  You would simply set the "Cache Entry Identifier" to "${fragment.identifier}-${fragment.index}". Then only the first occurrence of a FlowFile with unique fragment.index+fragment.identifier will make it in to the connection queue for your MergeContent processor.  Then the Defragment merge strategy will work successfully giving you what I believe you are looking for here.   Hope this helps with your use case, Matt ... View more

ceph is not supported and i don't know if they really follow the s3 interface.   try latest nifi 1.11.4 and use basic s3 mode ... View more