@DivyaKaki    The exception implies that the complete trust chain does not exist to facilitate a successful mutual TLS handshake between this NiFI and the target NiFi-Registry.   NiFi uses the keystore and truststore configured in its nifi.properties and NiFi-Registry uses the keystore and truststore configured in its nifi-registry.properties files. Openssl can be used to public certificates for the complete trust chain: openssl s_client -connect <nifi-registry-hostname>:<port> -showcerts   openssl s_client -connect <nifi-hostname>:<port> -showcerts for each public cert you will see: -----BEGIN CERTIFICATE----- MIIESjCCAzKgAwIBAgINAeO0mqGNiqmBJWlQuDANBgkqhkiG9w0BAQsFADBMMSAw HgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEGA1UEChMKR2xvYmFs U2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAeFw0xNzA2MTUwMDAwNDJaFw0yMTEy MTUwMDAwNDJaMEIxCzAJBgNVBAYTAlVTMR4wHAYDVQQKExVHb29nbGUgVHJ1c3Qg U2VydmljZXMxEzARBgNVBAMTCkdUUyBDQSAxTzEwggEiMA0GCSqGSIb3DQEBAQUA A4IBDwAwggEKAoIBAQDQGM9F1IvN05zkQO9+tN1pIRvJzzyOTHW5DzEZhD2ePCnv UA0Qk28FgICfKqC9EksC4T2fWBYk/jCfC3R3VZMdS/dN4ZKCEPZRrAzDsiKUDzRr mBBJ5wudgzndIMYcLe/RGGFl5yODIKgjEv/SJH/UL+dEaltN11BmsK+eQmMF++Ac xGNhr59qM/9il71I2dN8FGfcddwuaej4bXhp0LcQBbjxMcI7JP0aM3T4I+DsaxmK FsbjzaTNC9uzpFlgOIg7rR25xoynUxv8vNmkq7zdPGHXkxWY7oG9j+JkRyBABk7X rJfoucBZEqFJJSPk7XA0LKW0Y3z5oz2D0c1tJKwHAgMBAAGjggEzMIIBLzAOBgNV HQ8BAf8EBAMCAYYwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMBIGA1Ud EwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFJjR+G4Q68+b7GCfGJAboOt9Cf0rMB8G A1UdIwQYMBaAFJviB1dnHB7AagbeWbSaLd/cGYYuMDUGCCsGAQUFBwEBBCkwJzAl BggrBgEFBQcwAYYZaHR0cDovL29jc3AucGtpLmdvb2cvZ3NyMjAyBgNVHR8EKzAp MCegJaAjhiFodHRwOi8vY3JsLnBraS5nb29nL2dzcjIvZ3NyMi5jcmwwPwYDVR0g BDgwNjA0BgZngQwBAgIwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly9wa2kuZ29vZy9y ZXBvc2l0b3J5LzANBgkqhkiG9w0BAQsFAAOCAQEAGoA+Nnn78y6pRjd9XlQWNa7H TgiZ/r3RNGkmUmYHPQq6Scti9PEajvwRT2iWTHQr02fesqOqBY2ETUwgZQ+lltoN FvhsO9tvBCOIazpswWC9aJ9xju4tWDQH8NVU6YZZ/XteDSGU9YzJqPjY8q3MDxrz mqepBCf5o8mw/wJ4a2G6xzUr6Fb6T8McDO22PLRL6u3M4Tzs3A2M1j6bykJYi8wW IRdAvKLWZu/axBVbzYmqmwkm5zLSDW5nIAJbELCQCZwMH56t2Dvqofxs6BBcCFIZ USpxu6x6td0V7SvJCCosirSmIatj/9dSSVDQibet8q/7UK4v4ZUN80atnZz1yg== -----END CERTIFICATE----- Above is just example public cert from openssl command against google.com:443 You will need to make sure that every certificate in the chain when run agains NiFi UI is added to the truststore on NiFi-Registry and vice versa. You'll need to restart NiFi and NiFi-Registry before changes to your keystore or truststore files will be read in. Hope this helps, Matt ... View more

@Gubbi    Since you comment that NiFi starts fine when you remove the flow.xml.gz, this points at an issue with loading /reading the flow.xml.gz file.  I would suggest opening you flow.xml.gz in an xml editor and make sure it is valid.  When NiFi starts it loads the flow.xml in to heap memory.  From then on all changes made within the UI are made in memory and a flow.xml.gz is written to disk.  I would be looking for XML special characters (http://xml.silmaril.ie/specials.html) that were not escaped in the XML that written out to the flow.xml.gz.  Manually remove or correct the invalid flow.xml.gz may work. You may get lucky going back and trying multiple archived copies of flow.xml.gz files.  Perhaps one still exist prior to the config change that was made that introduced the issue? Note: Be aware that starting NiFi without the flow.xml.gz will result in loss of all queued FlowFiles. Hope this helps, Matt ... View more

@NY    There are several bugs related content repository failing to clean-up under specific conditions. Based on your description and version of HDF, the most likely bugs are the following: https://issues.apache.org/jira/browse/NIFI-6846 https://issues.apache.org/jira/browse/NIFI-5771   Both of the above would result in cleanup occurring only on NiFi Restart once the condition occurs. If you are using the VolatileFlowFileRepository implementation which is uncommon, you may be hitting: https://issues.apache.org/jira/browse/NIFI-6236 While you are not hitting below jira because it was only introduced in Apache NIFi 1.9.1 (HDF3.4.0 - removed) and fixed in HDF 3.4.1, sharing for completeness here: https://issues.apache.org/jira/browse/NIFI-6150 All the above are addressed, except NIFI-6846 are addressed in HDF 3.4.1 (Apache NiFi 1.9.2). All above are fixed in HDF 3.5.1 (Apache NiFi 1.11.4). I recommend upgrading to HDF 3.5.x (latest) Hope this helps, Matt ... View more

@anil35759  Your observations are exactly how Nifi is designed to work when importing version comtrolled flows from NiFi-Registry.   The first time you "deploy" a flow from NiFi-Registry it will add all the Controller Services (CS) utilized by the newly imported flow's components.  Subsequent version changes will not result in same behavior unless there is a new CS as part of the new version. So if you then deploy another version controlled flow that uses the same CS that the other already imported flow uses, you will end up with a second set of CS in the target NiFi.  There is not cross dependency on separately version controlled flows in NiFi-Registry. So each version controlled Process Group (PG) in NiFi will contain the complete set of referenced CS.  Post import of a CS to target NiFi, you may make changes to the CSs imported.  It would be dangerous for NiFi to assume that when importing another or the sane flow again from NiFi-Registry that it should reuse existing CS within the target NiFi.  Matt ... View more

@johannes_meixne    The only place you would use "{0}" would be in the ldap-provider used for user authentication.  The {0} gets replaced with the username entered in the login UI.  The LdapUserGroupProvider however is used by the nifi authorizer to assemble a list of users and group strings (along with the association/membership between those user and groups) which can be used to assign authorizations against.  The LdapUserGroupProvider executes on NiFi startup and then on a configured schedule to refresh the synced users and groups.  Since it takes no input you can not use "{0}" in the search filter. ... View more

@pazufst  You should make sure that you have both the core-site.xml and hdfs-site.xml files copied over to all your NiFi nodes.  Then make sure that these files are owned and accessible by the NiFi service user so that NiFi can read them. Here is another post that may help you more here: https://community.cloudera.com/t5/Support-Questions/NiFi-Hadoop-Configuration-Error/td-p/225448   Hope this helps, Matt ... View more

@anil35759    What is the best way to have controller services Globally or local for each flow? - Controller Services (CS) are setup at the Process Group (PG) level.  Sub process groups will have access (potentially depending on granularity of authorizations that have been setup) to CSs created at parent process group level.  So if you want to isolate what CS a particular group/team has access to you should create a different PG for each group/team in which they will create their dataflows.  Those dataflows will only hav access to the CSs created within their PG.  This prevents each team from messing with the configuration of a CS that may affect another team's dataflow. If controller services are configured globally ,what is the best way of mapping while deploying flow in another environment.   --- Not sure how you are currently "deploying" your flows, so this is a bit hard to answer. --- Generally speaking the deployment of flows from one NiFi environment to another is best handled via the NiFi-Registry service that is shared amongst all environments.  A dataflow is build within a PG and then that PG is version controlled to the NiFi-Registry (all referenced CS within the PG are included in that version controlled flow within NiFi-Registry).  Then within another NiFi environment, that flow can be imported from the same NiFi-Registry.  The added advantage to this deployment model is that new versions of that PG can be published to the NiFi-Registry and the other NiFi instances using that same registry will be made aware that a newer version is available and user can take action to update to the newer available version. Pro and cons of having controller services globally / locally. --- When you say "Globally", I assume you mean creating the CSs at the root PG level so that they are accessible by all sub PGs added within the UI?  Disadvantages here: 1. One team may make a change to the CS that they do not realize will affect another team's dataflow.  2. Making a change to a CS requires that the CS is disabled.  If team one disables the CS to edit it, any referenced component of the CS is also stopped.  After editing the team may enable the CS only and not restart the other teams referenced components. 3. If granular authorizations are setup user in one team may not be able to successfully edit a CS that is being actively used by a component in another teams PG that they do not have authorization to stop. So all the above really revolves around access controls and not much to do with the actual execution of your dataflows.  If you have one team that manages all your dataflows, I see no downside to having a single shared global CS except when you deploy each flow separately.  as part if deployment of a dataflow (how ever you do it), components will need the referenced CSs.  So if you may end up during deployment with multiple copies of the same CS with each separate dataflow deployment.     Hope this helps, Matt ... View more

@pazufst  There are a couple different things going on here. 1. It is the contents of the core-site.xml file you are pointing your putHDFS processor at that drives the specific security requirements of the putHDFS processor.  My guess is you will find kerberos setup within that core-site.xml file. 2. There are several Apache jiras for improvements to the the various AzureDataLakeStorage processors available within NiFi fo adding support for ADLS Gen 2.  These jiras are marked as resolved for Apache NiFi 1.12 which has not yet been released. https://issues.apache.org/jira/browse/NIFI-7259 https://issues.apache.org/jira/browse/NIFI-7334 https://issues.apache.org/jira/browse/NIFI-7336 https://issues.apache.org/jira/browse/NIFI-7340 Hope this helps, Matt ... View more

@Alexandros    As soon as you retrieve the json from which you later extract the sensitive values, those sensitive values are available/readable by anyone who has access to view the content of a FlowFile.  Even if you restrict user access so they can not view the FlowFile content, once you extract those sensitive values to FlowFile attributes they become exposed further.  There is currently no methods within NiFi for encrypting FlowFile attributes.  Doing so would also require any downstream processor in which you would want use those encrypted attributes to be able to understand that it is a sensitive value and decrypt it for use.  Bottom line here, is that the capability you are looking for does not exist within NiFi right now. This sounds like a new development opportunity/contribution maybe.  Perhaps a new NiFi controller service that handles pulling the credentials from the AWS Secret Manager and obtaining the JWT token without writing anything to the FlowFIle's attributes or content.  Then any processor you would want to use this new CS in would need to be extended to support the new capability.  I am not a developer myself, but this sound like non-trivial work. @alopresto might have some thoughts to add here. Thanks, Matt ... View more

@i_love_burger    I am not sure what configuration step you need help with. What is not "working"? What errors are you seeing when processor tries to execute? NiFi configured method for user authentication has nothing to do with how processors authenticate with external servers/resources.  Your user authentication and subsequent authorization simply controls what you as a NiFi user can see and do within the NiFi UI.  NiFi components like processors you add via the NiFi UI are not running/executing as your user.  All components are executed by the NiFi JVM service user.  Any authentication required is configured via the component itself. If your endpoint  httphandler running on port 8060 required mutual TLS authenctication via the TLS handshake, you will need to configure a SSLContextService that has both a keystore and a truststore. The keystore must contain a single "PrivateKeyEntry" that supports clientAuth and is capable of beig trusted by the target endpoint.  The Truststore used must contain the complete TLS trust chain for the target endpoint presented server certificate in the TLS hanshake. If only one-way TLS is required, all that is need is an SSLContextService with the above mentioned truststore. Using command: "openssl s_client -connect <servername>:<port> --showcerts"  is a good way to observe the server side handshake and obtain all the public certificates needed for a complete trusts chain in your truststore. Hope this helps, Matt ... View more

@LearnerAdmin    It is not clear to me what you are asking when you say " add NIFI CA in authorities".   Instructions on using the NiFi TLS toolkit can be found here: https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_toolkit   Using the Client/Server Tls Toolkit operational mode covered here: https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#client-server Will give you the ability to create a running NiFi CA authority "server" which will sign your NiFi node certificates created using the "client" mode.   Thanks, Matt ... View more

@100    Please share with me the exact NiFi processors in use here. Are you saying you have an external service listening on 8060 that you use a processor like InvokeHTTP to consume from that rest-api? NiFi authentication through the ldap-provider is only used for authentication to NiFi's rest-api endpoints.  It cannot be used for authenticated access to endpoint established by a NiFi processor(s).  Those processors typically only support TLS based authentication.    I am still not very clear on what you have setup here.  Screenshots of list of processors being used would be helpful.  Each processor runs its own code and does not typically have access to the NiFi core capabilities.   Hope this helps, Matt ... View more

@melvint    You might look in to using the "HashContent" and "DetectDuplicate" processors.  You can create a HASH for the content of each of your FlowFIles and use DetectDuplicate to see if a FlowFiles hash matches a previous hash that already processed.  If so, the duplicate is routed out of your regular dataflow path so you don't sent it HBase.   Hope this helps, Matt ... View more

@SamCloud    Not really how the processor is designed to work.  It order for a FlowFile to get outed to the duplicate path the distributed map cache must already have a matching entry in it.  All i can think for you to try is setting up a second distributedMapCache server where you use detect Duplicate a second time.  Those FlowFile routed down the duplicate connection the first time are used to populate the second cache server.  Then all your FlowFiles routed to non duplicate path originally are checked against that second distributedMapCacheServer.  Only thing here is you hav a bit of a race condition since you need to make sure the FlowFiles on the duplicate path are processed before those on the non-duplicate path.  For that you might want to use maybe the wait and notify processors. Wait on non duplicate path and the notify on after the detectDuplicate on the duplicate path.  You can see where this gets very complicated.  You may need to develop something custom here.   Matt ... View more

@100    I think more detail is needed here in order to help you with your query. Which processor did you configure with the SSLRestrictedContextService Controller Service? How was that processor and the controller service configured? What is the rest-api call you are making from Postman? What response to the rest-api call are you getting?   Thanks, Matt ... View more

@ppbirajdar    My next thought would be to just drop the cloned FlowFile right before the MergeContent since you intent is not to reassemble the exact same original file.  Your intent, if i understand correctly, is to just create one file that contains 1 FlowFile with each unique fragment index.   If that is the case, perhaps all you need is a detectDuplicate in front of your mergeContent that uses a combination of Fragment.identifier and fragment.index for determining if there is a duplicate FlowFile to drop.  You would simply set the "Cache Entry Identifier" to "${fragment.identifier}-${fragment.index}". Then only the first occurrence of a FlowFile with unique fragment.index+fragment.identifier will make it in to the connection queue for your MergeContent processor.  Then the Defragment merge strategy will work successfully giving you what I believe you are looking for here.   Hope this helps with your use case, Matt ... View more

@vivek12    Your last statement is confusing to me. Are you saying only the invokeHTTP processor is stopped after a NiFi restart?   This implies that someone stopped it or the flow.xml.gz is not getting updated with that processors last known state.  I'd inspect what is written in your flow.xml.gz on each node to make sure they all show the state as running. The last known state of a processor is not something that is checked as node join a cluster.  Nodes joining are checked to make sure their flow.xml matches the elected cluster flow and if it does, the node can join.  That node will then be told to start those components which are running in the cluster. When you restart the entire cluster each node presents its flow.  First node has Flow x and gets 1 vote, next node flow is checked and it matches exactly, that flow gets 2 votes.  Next node comes in and his flow is not exactly the same, so it gets 1 vote.  The flow with the most votes becomes the cluster flow.  If you have an even number of nodes (for example 4 nodes) and 2 node's flows get a vote and other 2 node's get a different vote.  since you have 2 vs 2, NiFi will end up picking one at random.  My concern here is that some node(s) have last known state as stopped while another is running.  So sometimes with a complete restart of your cluster you end up starting the flow with the stopped state on this processor. The other possibility is this invokeHTTP processor is failing validation on some node on startup and resulting in processor being stopped. Have you tried copying the flow.xml.gz from one node to all the other nodes?   Hope this helps, Matt ... View more

@SamCloud    The intent of the detect duplicate is to allow the first FlowFile through and then route all other FlowFiles with same cache entry identifier to duplicate.  So when a FlowFile reaches this processor it checks the Distributed Map Cache for the Cache Entry identifier, if it does not exist it is added to the cache and the FlowFile is routed to non-duplicate.  If it is found, then FlowFile is routed to duplicate. Based on the description you provided, it is working as intended/designed. Hope this helps, Matt ... View more

@psilvarochagome    The new product repositories are accessible only with a valid credentials.  Within HDF, the HDF 3.5.x + releases are the he first to require account access to download the binaries. You should reach out to your Cloudera Account team if you have trouble with your credentials.   Hope this helps, Matt ... View more

@ppbirajdar    If you have multiple FlowFiles with the same fragement.index and fragment.identifier, this indicates that at some point in your dataflow this FlowFIle was likely cloned (Same FlowFile routed to multiple relationships) or split (two FlowFiles created from one).  For example you dragged the success relationship twice.  Each of those FlowFiles will have however different UUID assigned to them.  You could use NiFi Provenance to look at the lineage of these FlowFIles to see at what point in you flow the clone was created.  In your case a CSV record that gets split into two FlowFIles (one with invalid records and another with some valid records) is going to be your issue since you route them to same destination processor. You didn't share your MergeContent processor configuration, but keep in mind the following: 1. A processor executes based on its configured run Schedule.  The default run schedule ins always 0 secs which means execute as fast as it can. This means a processor will execute and once it completes that execution, it will execute again. (If at execution there is no work to do, it will yield for a period of time before next execution is allowed to avoid excessive CPU usage when no work exists). 2. At time of Execution the MergeContent will look at what new FlowFiles since last execution and allocate those FlowFiles to bins.  If at end of that execution a bin meets the minimums configured, the bin will be merged (Mins don't matter if using Defragment merge strategy).  3. When using Defragment, the expectation is that all FlowFiles with same Fragment.identifier also have same Fragment.count and that only one FlowFile exists in a bin for each fragment.index value. It would be impossible to reassemble to original FlowFile since you have multiple FlowFiles now with the same fragment.identifier and fragment.index values.  The real question is whether you are actually trying to reassemble the exact original FlowFile or simply create a single FlowFile for which to trigger downstream processing? If the latter is the case, don't use Defragment merge strategy.  Use Bin-Packing Algorithm and set fragment.identifier as the "Correlation Attribute Name".  Since you really have no idea how many fragments there will be at this point, set your mins number of entires to a value large enough it would never be reached for a single fragment.identifier and then set the "max bin age" to a value high enough that you fee comfortable all fragments would have made it to the MergeContent.  Max.bin.age will force a bin to be merged at x amount of time after first FlowFile was allocated to that bin even if mins were not met. Also make sure you have configured n+1 number of bins, where "n" is the number of expected unique fragment.identifiers you expect to have at any one being binned by the MergeContent processor. Hope this helps, Matt ... View more

@abhinav_joshi  This is something that only affects the Apache NiFi releases only. The removal of some nars is documented in the Apache NiFi migration guidance wiki here: https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance   "Starting in 1.10,  the following nars were removed from the default convenience binary.  These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, druid-controller-service-api-nar, druid-nar, other-graph-services-nar.  You can still get them from the various artifact repositories and use them in your flows but we cannot bundle them due to space limitations by default." For Apache NiFi users, I recommend that users create a custom NiFi lib directory by adding the following property to their nifi.properties file:   nifi.nar.library.directory.<unique-string>=/<path to>/custom-lib1 for exmaple: nifi.nar.library.directory.NiFi-1-10-nars=/nars/nifi-1-10-lib   You can add as many custom lib directories as you want.  Then place any nars that are noted as removed via the migration guidance documentation into one of these custom lib paths before upgrading.  Then as part of your upgrade process add the above property to your nifi.properties file. Then later if additional nar bundles are deprecated, you can create another custom lib dir or just add those nars to your existing custom lib directory before upgrading.  This allows you to avoid the downtime you encountered.   The Cloudera releases of HDF and CFM include are built with all the nars by default currently as they are not affected by space limitations. Hope this helps, Matt ... View more