@dontiffjr  The exception in your browser  ERR_CERT_AUTHORITY_INVALID typically means that the trust chain for your NiFi's serverAuth certificate is not trusted by your browser.  You should see an option in the browser to "proceed to ...".  If you click on that, can you get to the NiFi UI? You can also use openssl command to inspect the server hello coming from your NiFi and obtain the public cert for your NiFi server's certificate.  You can load those public certificates into you browser trust. openssl s_client -connect <nifi-hostname>:<nifi-port> -showcerts Next thing to look at would be the contents of your certificate. <path to java>/bin/keytool -v -list -keystore <path to>/keystore.jks You'll want to make sure it contains: 1. A DN that does not contain wildcards 2. ExtendedKeyUsage (EKU) with both clientAuth and serverAuth 3. SubjectAlternativeName (SAN) with entry that matches the hostname of the server on which it is being used. 4. verify issue and expiration dates for certificate and that server clock and your local client machine where you are using browser has same date and time. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt   ... View more

@Griggsy  If you can share a source json, it may help with more specific guidance in the community. Thanks, Matt ... View more

@Neil_1992    The stack trace indicated your NiFi is running out of heap memory while loading your flow.xml.gz during startup? What version of NiFi are you using?  (Have you tried upgrading?) What version of Java is being used? (Have you tried upgrading?) Does your flow.xml.gz contain a lot of NiFi templates? Have you tried increasing the xms and xmx setting in the NiFi bootstrap.conf file? I suspect if you renamed your flow.xml.gz file that your NiFi would startup just fine.  Since i am guessing you do not want to lose your flow, I would increase the heap memory allocation for your NiFi.  Once up, I would look for way to reduce the heap usage of your dataflow(s) by removing old unused dataflows, and downloading templates so they are preserved outside of NiFi, and then delete templates inside of NiFi. If you found this response assisted with your query, please take a moment to login and click on " Accept as Solution " below this post. Thank you, Matt ... View more

@Ilaya    This is not an issue with NiFi service itself.  Chrome for some reason does not like the new certificates being used. I would start by comparing your old certificate with the new and make sure things like ExtendedKeyUsage (EKU) (needs clientAuth and serverAuth) and SubjectAlternativeName (SAN) entries are compete and accurate.  Make sure that the complete authority trust chain for your new certificate is present in you browser. Perhaps this resource maybe helpful as well: https://thegeekpage.com/bad-ssl-client-auth-cert/   If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Clarfim @tiagot    Neither NiFi or NiFi-Registry support creating locally managed users for the purpose of authentication in to services. NiFi provides numerous methods for authenticating a user/client.  The default which is always enabled is client/user based authentication via a Mutual TLS handshake. Other methods which can be configured in addition toTLS can be found here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication NOTE: As of Apache NiFi 1.14, a new Single User authentication method was introduced so that NiFi could be secure by default out-of-the-box.  If you choose to use this provider, it will be the ONLY user that can access this NiFi and will have full admin access.  If full multi-user access is desired, you must use another method of user authentication.   The "Users" UI that you see in NiFi is used so that NiFi authorization policies can be assigned to users and group strings. It is not used for creating local users with passwords within NiFi that can be used to authenticate into NiFi.  It also will not be present in UI if using the Single User authentication provider mentioned in previous note. When you configured the initial admin user string, it is used to by the authorizers.xml (depending on configuration) to initially construct a users.xml (contains user strings and group strings along with established association between them) and authorizations.xml (contains policies assigned to user and group strings) file.   Keep in mind that if the users.xml and authorizations.xml files already exist, they will not be updated if you make changes to your initial admin.  They are only generated f these files do not already exist. Above example that results in a users.xml if you are using the file-user-group-provider or legacy FileAuthorizer provider in your authorizers.xml. None of the other providers will produce a users.xml and none of the other providers will allow you to manually add new user strings or group strings.  The other providers are all used to pull user and group strings from an external source.  The File-Access-Policy-provider is used to construct the authorizations.xml file.  The Initial admin string set in this provider allows NiFi to seed the authorizations.xml file when generated with the NiFi policies needed for that user to function as an Admin (reminder: file is only generated if it does not already exist). More info on setting up authorization in NiFi can be found here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization   If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Hafiz    The ExtractText processor will evaluate a Java regular expression containing capture group against the inbound FlowFile's content. Then creates FlowFile attributes by processor dynamic property name that is assigned the value from the capture group from that Java regular expression. Above would result in FlowFiles with attributes like:   Things to keep in mind. SplitText takes the inbound FlowFile and splits it in too many FlowFiles.  If you are producing a lot of splits from a single source FlowFile, it will have an impact of NiFi's heap usage during that process.  As each Split FlowFile is created, the FlowFile attributes/metadata fro each produced FlowFile (splits) is held in heap memory.  Once all splits are created, all those produced Split FlowFiles are committed to the downstream relationship. One on the relationship, NiFi can then swap as needed out of heap memory.  NiFi does this to avoid data duplication.  Let's say you have a split that is in progress and NiFi dies.  Since nothing has been committed to a downstream relationship yet, when NiFi is brought back online, it will reprocess the original FlowFile.  You can reduce heap usage by splitting your source File multiple times if it is large (more than 20,000 - 50,000 splits).  For example, split by every 5,000 lines in first SplitText and then by every 1 line in second SplitText. While NiFi does not hold FlowFile content in heap memory (Some processor will load content in to heap to execute on that content), FlowFile attributes/metadata is held in heap memory.  So the more attributes/metadata exists on a FlowFile, the more heap that FlowFile is going to use.  FlowFiles are held in connection between processor components.  NiFi has a connection swap threshold that is applied per connection.  The default is to produce swap files that contain 10,000 FlowFiles each (these swap files are for FlowFile attributes/metadata and not content since it is not always held in heap). So swap default set in nifi.properties file is 20,000.  This means the first swap file for a connection is generated connection reaches 20,000 queued FlowFiles on one node (if multi-node NiFi cluster, swap is per node and not across all nodes) Just keep above in mind when designing dataflows where you are splitting/merging, creating a lot of FlowFile Attributes, or creating FlowFile attributes with large values. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Bakho @tiagot  NiFi 1.14.0 is the first Apache NiFi release that starts up secured by default: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.14.0 NOTE: when using the new Single-User-Provider, NiFi does not support setting new policies or adding additional users and groups via the UI.  The "single user" by default is authorized for what is needed to build dataflows.  So first make sure if you want to support multiple users, you configure your NiFi to use a different login provider or no login provider and use user/client certificates to authenticate via TLS. If you have already done the above, my guess here is that NiFi was launched for the first time before your initial admin identity was added/set. The NiFi authorizers.xml controls authorization of authenticated users. The default setup utilizes the following providers: <userGroupProvider> <identifier>file-user-group-provider</identifier> <class>org.apache.nifi.authorization.FileUserGroupProvider</class> <property name="Users File">./conf/users.xml</property> <property name="Legacy Authorized Users File"></property> <property name="Initial User Identity 1"></property> </userGroupProvider> <accessPolicyProvider> <identifier>file-access-policy-provider</identifier> <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class> <property name="User Group Provider">file-user-group-provider</property> <property name="Authorizations File">./conf/authorizations.xml</property> <property name="Initial Admin Identity"></property> <property name="Legacy Authorized Users File"></property> <property name="Node Identity 1"></property> <property name="Node Group"></property> </accessPolicyProvider> <authorizer> <identifier>managed-authorizer</identifier> <class>org.apache.nifi.authorization.StandardManagedAuthorizer</class> <property name="Access Policy Provider">file-access-policy-provider</property> </authorizer> The file-user-group-provider creates the users.xml file which holds your local created user and group associations (you would set your initial admin in the "Initial User Identity 1" property. The file-access-policy-provider creates the authorizations.xml file and generates authorizations needed for the configured "Initial Admin Identity" property identity. NiFi will only generate these files if they do NOT already exist.  If your NiFi was started before you set the properties I described above for both "Initial User Identity 1" and "Initial Admin Identity", the originally generated users.xml and authorizations.xml files would contain no users or authorizations. I suggest verifying the configuration in your authorizers.xml file, removing the existing users.xml and authorizations.xml files, and then restart your NiFi.  If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt   ... View more

@Jarinek    As part of changing the value of a parameter, all the components that are configured to use that parameter must be stopped and restarted.  Some components like controller services have dependent components like processors. So to stop such a controller service, the dependent processor must be stopped first.  When NiFi requests that a component stop, it transition in to an intermediate stage fo "stopping" where NiFi asks the thread to exit.  The UI would reflect component as stopped or disabled even though it is still in the transition stage. But that does not always guarantee a component will ever complete the transition from "stopping" to stopped.  Some client libraries used by components and not written by NiFi developers do support interrupting running threads.  In scenarios like this the the "stopping" thread kicked off by NiFi stays active against that component until the active processor thread finally exits (if it ever does).  And as i said that specific component thread never completing is out of NiFi's core control.  Restarting NiFi is only way to kill any threads that do not gracefully exit.   Things you can try here: 1. When you try to enable or delete a controller service, I'd expect the exception to provide the component UUID and host where it claims it is running.  You could go to the cluster UI and manually "disconnect" the node where it is still running.  Then open a web browser to that specific, now disconnected" node's UI.  Use the UUID of the component to locate that component and check its status.  Try disabling it and if that does not work, try restarting that node only.  If disabling the component worked, go back to UI of another node still in cluster and via the cluster UI request that the disconnected node reconnect.  While the exception might point out one host as the issue, it may be more than one host. The UI simply returns the first exception. 2. If above is not successful, have you tried restarting the entire cluster? 3. You could also inspect the the flow.xml.gz from each host to make sure they match.  Component state (started, stopped, enabled, and disabled) is retained in the flow.xml.gz, but is not included within the flow fingerprint used when connecting a node to the cluster.  As a connecting node is suppose to inherit the state from the cluster.  You could check each flow.xml.gz for the component UUID and see what its current recorded state is in the flow.xml.gz.  If you find one or more with a state of enabled/running and other nodes with an alternate state, simply copy the flow.xml.gz with the desired state to the nodes with the undesired state.  Make sure flow.xml.gz ownership and permissions are correct and restart your NiFi cluster. As far as to why a component never transitioned from state to another, that would require examining numerous thread dumps spaced apart by at least 5 minutes.  This would allow you to identify active thread making no progress over the course of those numerous thread dumps.  Then you would have to to dig in to those threads to see what they are waiting on or blocked by.  Sometimes there are limitation within some clients that NiFi has no control over. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@toninghaywi    The exception is implying that your NiFi is trying to connect to Zookeeper server for state management and your zookeeper is using TLS; however, your NiFi has not been configured to be able to establish that secure connection needed to your zookeeper. Apache NiFi support for TLS enabled zookeeper server was added in Apache NiFi version 1.13. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#zk_tls_client https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#embedded-zookeeper-with-tls   The Apache NiFi jira related to this feature improvement are below: https://issues.apache.org/jira/browse/NIFI-7203 https://issues.apache.org/jira/browse/NIFI-7115 https://issues.apache.org/jira/browse/NIFI-7819  <-- This one is specific to state management If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt     ... View more

@ykonduri    What version of Apache NiFi or CFM or HDF are you using? Have you added any non-standard additional nars, jars, etc to NiFi's lib or extensions folders? Thank you, Matt ... View more

@HiSunny  The configured "run schedule" on a NiFi processor controls how often the processor is scheduled to execute. The "concurrent tasks" controls the parallel execution of a processor.  When a processor is scheduled to execute it will request a thread from the NiFi Max Timer Driven thread pool that will be used to execute the processor code.  If that thread is still active upon next scheduled Execution and not all concurrent tasks are in use yet, the processor can request another thread to execute in parallel.   When it comes to variables, If the processor property support NiFi Expression Language (NEL) with an evaluation using FlowFile attributes, then each FlowFile via FlowFile attributes can provide unique input to the execution.   You can see if a property supports Expression language by floating your cursor of the "?" icon next to each property name.  You'll want to make sure the scope of NEL support supports FlowFile attributes. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Yemre  If you have a support contract with Cloudera, I'd recommend opening a support case to assist with your issue here. Possible causes: 1. Unsuccessful Mutual TLS handshake with NiFi-Registry from the NiFi hosts resulting in NiFi node connection only being 1-way TLS connection and node treated as an "anonymous" user.  Anonymous users would could not proxy user requests and can not see anything except public buckets. --- Caused by missing complete trust chain on one or both sides of connection.  Truststore in NiFi-Registry contains complete trust chain for NiFi hosts keystore PrivateKeyEntry. --- Caused by PrivateKeyEntry not meeting minimum requirements (missing SAN with NiFi hostname,  missing EKU of clientAuth, and/or using wildcards are the most common) 2. NiFi-registry is configured with an identity mapping pattern in the nifi-registry.properties file that is matching on the DN from the the NiFi's client certificate presented in the mutual TLS handshake. The Identity mapping value  and transform is then being applied which alters the actual client string which must be then authorized for Proxy and buckets policies.   Hope this helps you, Matt ... View more

@Ankit13  I would still use Cron scheduling on the PutFile processor, but rather than just having it run once at say hour 7, I'd schedule it to run every second starting at hour 7.  That may it starts putting files at hour 7 and continues to put files all the way until 07:59:59.   Then it stops executing until the next day. http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html Hope this helps,  Matt ... View more

@sandip87  This statement is not clear to me: We had combined them to form one certificate and then follow steps in the NiFi documentation. Sounds to me like your trusts chain has an intermediate and root CAs in it. That means your truststore must have two trustedCert Entries in it.  One for intermediate CA and other for the Root CA. It sounds like you only have the root CA in your truststore.   If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Yemre  Authorizing your user is not enough. The NiFi nodes themselves need to be able to successfully authenticate via a mutual TLS handshake with the target NiFi-Registry.  Those nodes then need to be authorized to read all buckets and given read/write to proxy user requests. When a User authenticates in to NiFi, that user entity is authorized to perfrom actions based on authorizations in NiFi. When it comes to NiFi then talking to NiFi-Registry, The NiFi node is proxying request to the NiFi-Registry on behalf of the user authenticated into NiFi. Also background threads in NiFi just like the NiFi processors added to the canvas are not executing as the user authenticated in to NiFi.  So in the background NiFi connects to NiFi-Registry to check on current version controlled process groups to see of newer versions exist. While you are granting your NiFi nodes the ability to read all buckets, the NiFi users should be given read and write authorizations to the specific buckets that that user is going to sue to version control their Process Group. @Yemre    The ability to dynamically fetch secrets/passwords form an external source is not something that exists currently. Doing so would require modification with the every component class that uses sensitive properties. There is some progress in this path however: https://issues.apache.org/jira/browse/NIFI-5481   This new feature handles pulling secrets from an external vault, but is a NiFi core level feature and does not extend in to individual flow component level. I recommend raising an Apache NiFi Jira with your specific request.  https://issues.apache.org/jira/projects/NIFI/ If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Yemre    The ability to dynamically fetch secrets/passwords form an external source is not something that exists currently. Doing so would require modification with the every component class that uses sensitive properties. There is some progress in this path however: https://issues.apache.org/jira/browse/NIFI-5481   This new feature handles pulling secrets from an external vault, but is a NiFi core level feature and does not extend in to individual flow component level. I recommend raising an Apache NiFi Jira with your specific request.  https://issues.apache.org/jira/projects/NIFI/ If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@sandip87    The following properties within the nifi.properties file will tell you where your NiFi's keystore and truststore files are located: 1.  nifi.security.keystore 2. nifi.security.truststore   You can use the java keytool command to see the verbose details of the content of these two keystores: <path to java JDK>/bin/keytool -v -list -keystore <keystore/truststore>   Once you inspect the these to make sure the contents are good, then you need to make sure you can successfully authenticate your user in to your NiFi. By default once NIFi is secured, the only method to authenticate a user/client is via a mutual TLS handshake which means your user needs to have a certificate loaded in the browser. Optionally you can add additional user authentication methods if you want. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication     ... View more

@Sridhara47    A status code 302 means there was a redirect in the response. I'd suggest inspecting the NiFi app.log for this exception and take a look at the full stack trace if it exists to see what the issue may be. Hope this helps, Matt ... View more

@sandip87  Without the detailed output of your NiFi keystore and truststore and the of your client certificate you are using to authenticate yourself to NiFi, it would be difficult to say exactly where your issue is.  I am leaning towards and issue with your company issued certificates because you stated the this same NiFi worked fine when using certificates and keystores generated by NiFi's TLS toolkit. For NiFi's keystore (configured in nifi.properties file), make sure the following are correct: 1. keystore contains 1 and only 1 PrivateKeyEntry 2. Make sure the PrivateKeyEntry ExtendedKeyUsage (EKU) contains clientAuth and serverAuth 3. Make sure that the PrivateKeyEntry contains a SAN entry that matches the hostname of the host where NiFi is running. For the user certificate loaded in the browser being used to authenticate with this NiFi: 1. verify certificate issuer.  Is it an intermediate CA or the root CA? 2. Verify the NiFi's truststore.jks (configured in nifi.properties file) contains aTrustedCertEntry for the complete trust chain that goes with your certificate and the certificate found in the keystore.jks.  A complete trust chain means that the truststore has the public keys for the issuer of each of the above certificates and if that issuer is and intermediate CA, you also have the public certificate for the CA that signed that intermediate CA in the truststore.   You'll know when you have reached the root CA when the TrustedCertEntry has the same DN for both owner and issuer. Your browser must also contain the complete trusts chain for the certificates issued to your NiFi nodes. Once all the above is verified, clear your browser cache and site cookies. If you still have same issue and you are using Chrome browser, try typing "thisisunsafe" (which tells chrome to skip certificate verification on the certificate presented from the NiFi instance) while the NiFi chrome tab is in focus.  If this works and allows you to proceed, this again points at a trust issue between your corporately issued certificate and your browser. Go back and verify structure/content of the NiFi keystore again. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@AnnaBea  Let me make sure I am clear on your ask here: 1. You have successfully split your source file in to 3 parts (header line, body line(s), and footer line). 2. You have successfully modified all three split files as needed. 3. You are having issues re-assembling the three split files back in to one file in order of header, body, footer using MergeRecord processor? With this particular dataflow design, the MergeRecord processor is not likely what you want to use.  You probably want to be using the MergeContent processor instead with a "Merge Strategy" of  "Defragment".  But to get these three source FlowFiles merged in a specific order would require some additional work in your upstream flow.  In order to use "Defragment" your three source FlowFiles all would need o have these FlowFile Attributes: fragment.identifier All split FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attribute fragment.index A one-up number that indicates the ordering of the split FlowFiles that were created from a single parent FlowFile fragment.count The number of split FlowFiles generated from the parent FlowFile   1. Add one UpdateAttribute processor before your RouteText and configure it to create the "fragement.identifier" attribute with a value of " ${UUID()}" and another Attribute "Fragment.count" with a value of "3".  Each FlowFIle produced by RouteText should then have these two attribute set on it. 2. Then add one UpdateAttribute processor to each of teh 3 flow paths to set the "fragment.index" attribute uniquely per each dataflow path.  value=1 for header, value=2 for body, and value=3 for footer. 3. Now the MergeContent will have what it needs to bin these three files by the UUID and merge them in the proper order. There are often times many ways to solve the same use case using NiFi components.  Some design choices are better than others and use less resources to accomplish the end goal. While above is one solution, there are others I am sure.  Cloudera's professional services is a great resource that can help with use case designs. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Ankit13  My recommendation would be to only automate the enabling/disabling and starting/stopping of the NiFi processor component that is ingesting the data in to your NiFi dataflow and leave all downstream processors always running, so that any data that is ingested to your dataflow has every opportunity to be processed through your dataflow to the end.  When a "running" processor is schedule to execute, but has no FlowFiles queued in its inbound connection(s), it is pauses instead of running immediately over and over again to prevent excessive CPU usage, so it is safe to leave these downstream components running all the time. Thank you, Matt ... View more

@galvinpaul1718  I 'd suggest verifying your download was good. Then remove the nifi-registry work directory before restarting. The work directory is rebuilt from the nifi-registry lib dir contents. Make sure you did not run out of disk space. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

@Apoo  The EvaluateJsonPath processor dynamic properties do not support NiFi Expression language, so being able to pass dynamic strings to these dynamic properties from FlowFile attributes is not possible.   The dynamic properties only support NiFi parameters. You may want to raise an Apache NiFi jira requesting adding NiFi EL support to these dynamic properties or even contribute the the open source code if you so choose. Thank you, Matt ... View more