Member since
07-30-2019
3467
Posts
1641
Kudos Received
1015
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 414 | 03-23-2026 05:44 AM | |
| 321 | 02-18-2026 09:59 AM | |
| 565 | 01-27-2026 12:46 PM | |
| 999 | 01-20-2026 05:42 AM | |
| 1315 | 01-13-2026 11:14 AM |
01-07-2020
01:49 PM
@venu413 If you open the NiFi summary UI (NiFi UI --> Global menu --> Summary), select the connections tab, locate this connection with the 54 queued flowfiles, and then click the cluster connection summary icon ( )to far right, Are all 54 queued FlowFiles on same node? Is anything being logged in the nifi-app.log on that node were these FlowFiles are queued? Any observed errors in nifi-app.log during startup if you restart this node? In your nifi.properties file, what values are configured for these properties: nifi.cluster.load.balance.comms.timeout= nifi.cluster.load.balance.connections.per.node= nifi.cluster.load.balance.host= nifi.cluster.load.balance.max.thread.count= nifi.cluster.load.balance.port= nifi.cluster.node.address= If recommend that both the "nifi.cluster.node.address=" and "nifi.cluster.load.balance.host=" have been configured uniquely per node in your cluster to the resolvable hostname for the given node. So if you node has a hostname of node1.mycompany.com, then this hostname should be used in both these properties in the NiFi running on that host. Restart is needed anytime you edit the nifi.properties file.
... View more
12-27-2019
02:42 PM
1 Kudo
Hello @Former Member Now that you have both NiFi and NiFi-Registry secured they will use TLS to authenticate with one another. NiFi-Registry does not initiate any connections to NiFi. NiFi will always act as the client talking to NiFi-Registry. 1. All 3 of your NiFi nodes must exist as users in the NiFi-Registry. 2. Any users who will be version controlling NiFi process Groups will need to exist as users in NiFi-Registry. 3. Your NiFi nodes must be authorized in NiFi-Registry for "Can proxy user requests" and read for "can manage buckets". Found by clicking on settings in NiFi-Regsitry UI, then selecting Users tab, and clicking pencil to right of each of your NiFi nodes. 4. You users must create a bucket(s) in NiFi-Registry and authorize your NiFi user(s) for read, write, delete on the bucket. From same setting UI click buckets tab, click "add bucket" then using pencil to left of bucket authorize your user(s). 5. From the NiFi UI, click on the global menu (upper right corner) --> Controller Settings --> Registry Clients tab. Click the "+" icon to add a new NiFi-Registry client. Provide the HTTPS://<nifi-regsitry-hostname:port> as the URL and a name of your choosing. Provide the keystores and truststores created for your NiFi and NiFi-Registry can support mutual authentication between these two services, you will be good to go. Otherwise check your nifi and nifi-registry app logs for any TLS handshake errors which would need to be resolved. Hope this helps you get going with NiFi-Registry, Matt
... View more
12-27-2019
02:25 PM
@stevenmatison I don't see any reason why you can not do this. Do NOT put the 1.10 nar in to NiFi's defualt lib directory. Instead create a new custom lib directory by adding a new property to the nifi.properties file: nifi.nar.library.directory.custom1=/<path>/custom-libs Copy the 1.10 parquet nars in to this directory and make sure the permissions and ownership are set correctly for this custom path and libs so that the NiFi service user can access them. Once NiFi is restarted, you should see both versions of the parquet processor available for adding to the NiFi canvas. Note: Adding additional versions of the same nars can make upgrading a bit more challenging. After copying the new nar in to custom nar lib, you now have both 1.9 and 1.10 versions of the NiFi components. Let's say later you upgrade to 1.11 when it becomes available for example. NIFi normally handles upgrading to new versions during startup (1.9 processors upgraded to 1.11), but in you case after upgrade you will have this custom lib nar 1.10 and 1.11 that cam with upgrade. Any 1.9 parquet components from your flow.xml.gz will become ghost processors on the canvas because NiFi does not have a version 1.9 and there are two options (1.10 and 1.11) so it picks neither. You would need to either drop ghost processors and add the 1.10 or 1.11 version in its place or manually edit the flow.xml.gz to change version number manually to desired available version. Hope this helps, Matt
... View more
12-27-2019
07:14 AM
@Former Member Since you are asking a new question unrelated to the question asked in the original subject, I kindly ask that you start a new question. Would be happy to help. Asking multiple questions in one thread makes a thread harder to follow for other users of this community forum. If you feel this question subject has been answered, please accept a solution provided to close out this thread. Thank you, Matt
... View more
12-26-2019
08:38 AM
1 Kudo
@Former Member Make sure you have configured "nifi.registry.security.needClientAuth=false". When not set it defaults to true. NeedClientAuth=true tells NiFi that in the TLS handshake it will "require" client to present a client side certificate. If one is not presented, the connection will just close and NiFi-Registry will never try any other authentication method. This property must be set to false in order for NiFi-Registry to support any authentication method other than TLS. Hope this gets you going, Matt
... View more
12-23-2019
10:24 AM
@Former Member Simply configuring the ldap-provider in the identity-providers.xml file will not result in NiFi-Registry using it. Make sure you have set the following property in the nifi-registry.properties file: nifi.registry.security.identity.provider=ldap-provider This tells NiFi to use the "ldap-provider" configured in that file. Also make sure the file is named "identity-providers.xml" and not "login-identity-providers.xml". NiFi-Registry uses the former while NiFi uses the latter identity providers filename. One other things to consider... If NiFi-Registry is configured to support Spnego: nifi.registry.kerberos.spnego.authentication.expiration=12 hours
nifi.registry.kerberos.spnego.keytab.location=
nifi.registry.kerberos.spnego.principal= Spnego auth will be attempted before any configured identity provider. So all it takes is to have Spnego enabled in your browser and NiFi-Registry to be setup to support Spnego auth and you will not see login page as well. If you do not have Spnego enabled in your browser, then this is not your issue because even if configured if browser does not return Spengo creds, NiFi-Registry will move on to next configured authentication provider. Hope this helps, Matt
... View more
12-23-2019
09:48 AM
@Boenu You will need to configure your HandleHttpRequest processor with a SSL Context Service in order to encrypt data in transit being sent to this processor from a client. This of course then means you client needs to be able to at a minimum to trust the server certificate presented by this SSL context service in the TLS handshake. The truststore you use in the NiFi SSL Context Service will only need to contain the public cert for your client or complete certificate trust chain for your client if you have configured your HandleHttpRequest processor to "Need authentication" in the Client Authentication property. Mutual Authentication is not needed to ensure encryption of data in transit. Hope this helps, Matt
... View more
12-18-2019
04:41 PM
@girish6 Anything you can do via the NiFi UI, can also be done using the NiFi Rest-api. Using the developer tools available ion most browsers is a great way to learn how the rest-api calls work. You can have the developer tools UI open while you perform similar actions on the canvas and then capture those rest-api request directly from the developer tools (most offer an option to "save as curl"). It can be challenging to build dataflows via the UI in terms of placement. While NiFi does not care if all your components are stacked upon one another at position 0,0, if you ever try to access the UI to modify a component that way later it would be nearly impossible as it would be one blob in the center of the canvas. You can find the rest-api docs embedded in your NiFi installation by going to the Global Menu in the upper right corner of your NiFi UI and selecting Help. Hope this helps, Matt
... View more
12-17-2019
07:04 AM
@saciya I am not sure why this is not working for you. I built a test flow and it is working fine for me. Here is how I have my ReplaceText processor component configured: I also used "shift+enter" to create a new line at the end of the header being inserted (note that a blank line "2" is shown) and the "Prepend" replacement strategy. I passed it the following test CSV content: 15-Dec-19,Baltimore,27
17-Dec-19,Baltimore,34 I then listed the queue on the success relationship connection outbound from the ReplaceText processor. From there I could click on "View Details" icon to far left of my FlowFile in the queue list and then click "VIEW' to see what the resulting content looked like. Here is what I see for my sample source content above: I am using Apache NiFi 1.9 Hope this helps, Matt
... View more
12-17-2019
06:30 AM
@PavelPrudnikov The specific error you are seeing in the bulletin produced on your PutDatabaseRecord processor is telling you that the processor is not able to find the schema you have configured in your CSVReader. Did you define a schema in one of the Schema Registry providers? Is the value assigned to the "schema.name" FlowFile Attribute on your inbound FlowFile an exact string match with the schema name in your Schema Registry provider? It may be helpful if you share a screenshot of the FlowFile attributes on your FlowFile in the queue to the putDatabaseRecord processor and share your CSVReader configuration as well. Have you tried just using the infer-schema option in the CSVReader Schema Access Strategy? Hope this helps, Matt
... View more