@JelenaS  You are correct that this sounds like an authorization issue. I recommend tailing the nifi-registry-app.log  and then perform the action of trying to version control a Process Group within NiFI's UI. How are you handling user authorization in your NiFi and NiFi-Registry? - File based authorization (users.xml and authorizations.xml) What identity.mapping patterns have you configured in your NiFi and NiFi-Registry? How are you authenticating your user that access both NiFi and NiFi-Registry? The only buckets that would be returned are those buckets for which the Authenticated user in NiFi has access to in NiFi-Registry. Keep in mind that the user/client strings in NiFi that are passed to NiFi-Registry must match exactly. Nodes will pass their full DN when they proxy the request on behalf of the authorized user.  The user string will passed as is.   That means identity mapping patterns will be applied on NiFi-Registry side against those NIFi DNs.  Resulting mapped value must match the client string add as a user in NiFi-Registry. The passed user string must match exactly (case sensitive) or it is treated as a different user.   Hope this helps, Matt   ... View more

@pjagielski  It is always helpful to share the exact NiFi version you are running as there may be known issues we can point you. Assuming here that you may be running latest Apache NiFi 1.12 release, my first thought may be related to this issue: https://issues.apache.org/jira/browse/NIFI-7992 While your content repo is not filling up, I would suggest inspecting you logs to see how often Content claims are being moved to archive.  A background thread then removes those claims as a result of your archive settings. Hope this helps, Matt ... View more

@Nyk  You'll need to provide more context around your use case to get a good answer.  It is not clear what you are trying to accomplish here. - Are you trying to build your own custom NiFi processor where you want to define some configurable properties as type sensitive? - Are you trying to use the "+" icon to add a new dynamic property to an existing NiFi processor and hide/encrypt the value set in that property?  In this case, that is not possible since all dynamic properties are nonsensitive. The use of invokeScriptedProcessor you saw was a recommendation made to sue it rather than ExecuteScript processor.  That is because you can define properties via the invokeScriptedProcessor custom script you write additional processor properties to be used within that processor only as sensitive. But looks like there may be some issues around that ability outlined here: https://issues.apache.org/jira/browse/NIFI-7012 Hope this helps, Matt ... View more

@te04_0172    It appears you have hit a known issue: https://issues.apache.org/jira/browse/NIFI-7954 https://issues.apache.org/jira/browse/NIFI-7831 Looks like these will be addressed in Apache NiFi 1.13 These fixes have already been incorporated in to the Cloudera HDF 3.5.2 release that is currently available. Hope this helps, Matt ... View more

@dzbeda    Seeing "localhost" in your shared log output leads to what may be the issue. When you configure the URL in the Remote Process Group (RPG), it tells that NiFI RPG to communicate with that URL to fetch the Site-To-Site (S2S) details.  Included in those returned details are things like: - number of nodes in target NiFi cluster (if standalone returns only one host) - The hostnames of those node(s) (in this case it looks like maybe localhost is being returned - Configured RAW port if configured - Whether HTTP transport protocol is enabled - etc... So when you RPG actually tries to send FlowFiles over S2S it is trying to send to localhost which results in itself rather than the actual target linux NiFi it fetched the S2S details from. When some properties are left unconfigured, NiFi returns whatever the OS resolves. I am going to guess your linux server is returning localhost rather than the actual hostname. You will want to verify your S2S configuration setup in the target NiFi (linux server): http://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#site_to_site_properties Try setting the "nifi.remote.input.host" to see if that helps you. Hope this helps, Matt ... View more

@venkii  User/client authentication via a user certificate and Authentication via login provider are handled completely differnetly Looking at your login-identity-providers xml, I see you have configured the following: <property name="Authentication Strategy">START_TLS</property> However, you have not configured any of the TLS properties in the provider.  Are you sure "START_TLS" is what you want to be using here?  Your ldap URL looks to be using just ldap:// and the default unsecured port.  If that is the case the "Authentication Strategy" should be set to "SIMPLE" instead of "START_TLS". The exception points to at an SSL handshake exception.  It implies that a server certificate returned by the ldap server did not match (IN either DN or SAN entry) the hostname used in your ldap URL configured in the login provider. So in this case you either need to switch to "SIMPLE" (if possible) or validate the server certificate being returned by your ldap server and setup needed TLS properties in your provider. Hope this helps, Matt ... View more

@dzbeda  Can you share a little more about your use case? NiFi does not expire data that is actively queued within connections between components added to the NiFi canvas.   So I am a bit curious on the "I don't want to lose data" statement you made. It is true that during times of "connectivity issues between the sites" that  NiFi FlowFile may accumulate within the connection queues resulting in more storage being needed to hold that queued data while you wait for the connectivity to restore, but still not a concern for "data loss" unless your ingest is using some unconfirmed transfer protocol like UDP.  NiFi's Site-To-Site protocol used by the Remote Process Groups uses a two phase commit to avoid dataloss. Backpressure settings on each connection can control how many FlowFiles can queue before the component feeding FlowFiles into the connection is o longer allowed to execute. So in an extended outage or high volume, backpressure could end up being applied to all connection from last component in your dataflow to the first component in your dataflow.   Default object thresholds are (10,000 FlowFiles or 1 GB of content size). Keep in mind these are soft limits.  Not advisable to simply set backpressure to some much larger value. I recommend reading following article: https://community.cloudera.com/t5/Community-Articles/Dissecting-the-NiFi-quot-connection-quot-Heap-usage-and/ta-p/248166 As far as what happens when the content repo(s) (NiFi allows you to configure multiple content repos per NiFi instance) are full, NiFi simply can not generate any new content.  So any component that tries to create new content (can be at ingest or via some processor that modifies the content of an existing FlowFile) will simply fail went it tries to do so with an out of disk space exception. This does not mean dataloss (unless as I mentioned your ingest or egress uses an unconfirmed protocol).  The component will simply try again until it is successful once disk space becomes available (For example when connectivity returns and data can be pushed out). Using good protocols would result on data remaining on source once backpressure is applied all the way back to your ingest based components. NiFi archiving has nothing to do with how long FlowFiles are kept in NiFi's dataflow connections.  Archiving holds FlowFiles after they have successfully been removed (reached point of auto-termination in a dataflow.   Archiving allows you to view old FlowFiles no longer queued or replay a FlowFiles from any point in your dataflow.  However, there is no bulk replay capability, so not useful for that. https://community.cloudera.com/t5/Community-Articles/Understanding-how-NiFi-s-Content-Repository-Archiving-works/ta-p/249418 Hope this helps, Matt ... View more

@SandeepG01    What the ERROR is telling you is that some component the references the parameter context failed to transition to a stopped state.   When a parameter context is defined in a component (NiFi processor for example), and change to the value assigned to that parameter would require the restart of those components so that the new parameter value is read in by the component. When a component like a NiFi processor is asked to stop, it first transitions into a state of "Stopping".  Once any active threads on that component complete the processor transitions to a "Stopped" state.  It as this point the component could be started again. So there are two possible scenarios that can happen here Since NiFi will not interrupt an active thread: 1. A component had a ling running thread that the restart process timed out waiting for it to transition to stopped (thread to complete and exit).  It is quite possible that this component did finally finish its  thread execution post this timeout and another attempt to apply the parameter context change would not be successful. 2. A component has a hung thread.  Meaning the active thread on the component is making no progress because it is either waiting indefinitely on some response action or it being blocked by some other thread.  In this case it would never complete its execution and transition to the "Stopped"state.  First you need to identify the processor with the hung thread (Processors show a small number in t he upper right corner when a thread is active).  Then you can take a series of thread dumps (./nifi.sh dump <dump-file-01>) and compare those thread dumps to see if the you have a thread stuck in the same state in every dump (shows no thread progress).  The only way to kill a hung thread is via a NiFi restart.   NiFi processors offer a terminate option, but that does not kill the tread.  It simply disassociates that thread with the processor (think of putting in an isolation box). The flowfile that was being processed by that thread is rolled back on to the incoming connection.  Should that terminated thread ever return anything (lets say it was not hung, but just a very long running thread), it output is just sent to null since it was terminated.  But again that thread if truly hung will not go away until NiFi restart even if you selected terminate on the component. You many also inspect the nifi-app.log for the ERROR exception to see if it is followed with a stack trace that may give more insight. Hope this helps, Matt ... View more