Member since
07-30-2019
3406
Posts
1622
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 151 | 12-17-2025 05:55 AM | |
| 212 | 12-15-2025 01:29 PM | |
| 145 | 12-15-2025 06:50 AM | |
| 264 | 12-05-2025 08:25 AM | |
| 427 | 12-03-2025 10:21 AM |
01-17-2020
02:05 PM
@JamesE You can handle this easily using a different set of Java Regular Expressions: .*action=(.*?) .*
.*srcip=(.*?) .*
.*timestamp=(.*?) .* If it is possible that any one of these fields may be the very last field in the content line, for this to work you would need to append a blank space to the end of the content using the ReplaceText processor before sending your FlowFile to your ExtractText processor. You need to have a blank space following each value so regex know where the value ends for each field. Your ReplaceText processor configuration would look like this: The "Replacement Value" is just a single space. Hope this helps, Matt
... View more
01-17-2020
01:51 PM
@venkii You need to generate a certificate for each of your NiFi nodes in your NiFi cluster. While you can create just one certificate (single DN) with multiple Subject Alternative Names (SAN) entries for each host, this is not security best practices. There is no point in setting multiple identities for the same DN string. For each "Initial User Identity <num>", the configured string is added as a user in the users.xml file for each "Node Identity <num>", the configured string has authorizations needed for NiFi nodes associated to that user's uuid in the authorizations.xml file. In order to access the NiFi UI after securing NiFi, your user/client will need to authenticate with your NiFi. By default the only configured user authentication method setup is via TLS certificates which requires you to to create a certificate for each user accessing NiFi. This certificate would be loaded in to your browser and not NiFi itself (the public certificate for your user's private certificate would need to be added to the NiFi truststore in order for NiFi to trust the cert). NiFi can also be configured to support kerberos, ldap, openId connect, etc alternative methods for user authentication. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Once a user is successfully authenticated, that user must be authorized before they can even see the NiFi UI. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization I understand you have already started down this path, but I recommend you carefully read above document links for more details.
... View more
01-17-2020
01:36 PM
@JamesE This is easy enough do using the splitContent processor: for the "Byte Sequence" property simply hit your spacebar to set a single space. If you found this answer addressed your question, please take a moment to accept to resolve this thread. Hope this helps, Matt
... View more
01-16-2020
11:16 AM
@Chada Make sure your PrivateKeyEntry being used by your NiFi has both "clientAuth" and "serverAuth" extended key usage. The NiFi-Registry URL configured in NiFi --> global menu --> controller settings --> registry clients tab will be: https://<nifi-registry-hostname>:<nifi-registry port> Your NiFi node(s) will need to exist in NiFi-Registry and be assigned the following special privileges: Read allow your NiFi nodes to read all buckets to see if new versions of existing version controlled flows exist. Proxy allows is needed because a NiFi node may proxy requests on behalf of the user authenticated in to NiFi and making NiFi-Registry requests. Your NiFi authenticated user in NIFi will need to exist in NiFi-Registry (exact string match) and be authorized for a bucket you must first create via the NiFi-Registry's UI as follows: If your NiFi user has not been authorized to any buckets you will see below error in NiFi when you try version control a process group: Once the user is properly given authorization to a bucket in NiFi-Registry, you will instead see: Only buckets for which your user is authorized will show in the bucket pull-down menu. Hope this helps, Matt
... View more
01-16-2020
10:23 AM
@JamesE The ExtractText processor is used to extract text from the content of the FlowFie using a Java Regular Expression and insert that extracted text in to FlowFile attributes. So using your FlowFile content example here: srcip=10.10.10.10 timestamp=152532431 action="denied" What is your desired end result? Three separate FlowFileAttributes? One for each "word" (srcip, timestamp, and action). Assuming above, you would add three new properties to the ExtractText processor (one for each extracted value) as follows: For each dynamic property added via the "+" icon, The property name becomes the FlowFile attribute name and the resulting string from capture group within the Java regular expression becomes the value assigned to that new FlowFile attribute. Hope this helps, Matt
... View more
01-16-2020
09:36 AM
@justenji That bug was actually introduced through a change that went in to NiFi 1.8 that pushed included the enabled/disabled state to registry. But NiFi registry shipping at that time had not yet been updated to store this new parameter. Do you still see the issue even after upgrading to NiFi-Registry 0.5?
... View more
01-16-2020
09:20 AM
@venkii During NiFi startup as NiFi loads the authorizers.xml it will use the configured "ldap-user-group-provider" to seed NiFi will a list of users and groups based upon that providers configuration. The error is telling you that the connection attempted based on your configuration timed out to ldap.ad.xx.xx:389. I would verify that every node in your NiFi cluster can resolve your ldap hostname and and there are no network firewalls blocking any of your nodes from connecting. As far as your ldap-user-group-provider configuration goes... 1. Perhaps adjust your Connect and Read timeouts to see if that helps. 2. Your Sync interval is aggressive at every 20 minutes. Are new users and groups commonly added to ldap that often? 3. Not really sure how many users are part of your two groups, but if it is a lot you will want to configure a page size (500 for example) 4. You did not set your "User Group Name Attribute" which based on your other configuration should be "memberOf" Additionally... Why are all your Initial user identities and Node Identities set to "CN=Nifi_Admin,OU=NIFI" . These should be unique DNs for each of the nodes in your cluster. Your initial Admin Identity in the file-access-policy-provider is set to CN=admin, OU=NIFI. Since this initial Admin identity is also not configured in the file-user-group-provider, it is expected that this user is one of your ldap users. Problem here is that your ldap-user-group-provider is using sAMAccountName value as the user string and that value will not contain a full DN like you have configured. Anytime you start NiFi without existing users.xml or authorizations.xml files, they will be built by the authorizer. If they already exist they will not get modified or replaced. So if you are still editing the authorizers.xml file to get your setup working, you may need to delete these two files from each node before each restart attempt if you make any changes to the Initial User Identities, Node Identities, and/or Initial Admin Identity. NOTE: authorizers.xml and authorizations.xml files are two totally different files. Make sure you do not delete the wrong one. Hope this info is helpful, Matt
... View more
01-16-2020
06:51 AM
@venkii Your first authorizers.xml file was not configured correctly which prevented your startup. You "authorizer" (ranger-provider) was looking for the "file-access-provider" (which exists) and is configured to look for the "composite-configurable-user-group-provider" (which does not exist). The file-access-provider should have been configured to look for the "file-user-group-provider" which you do have. As far as the Ui not be accessible once you switched to an entirely different authorizer... The UI is the last thing that becomes available during the startup process. Until you see the following lines output in the nifi-app.log, the application has not completed startup yet. <timestamp> INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs:
<timestamp> INFO [main] org.apache.nifi.web.server.JettyServer <a href="https://ip-172-31-38-192.us-west-2.compute.internal:9696/nifi" target="_blank">https://ip-172-31-38-192.us-west-2.compute.internal:9696/nifi</a> Hope this helps, Matt
... View more
01-10-2020
05:14 AM
@justenji What version of NiFi-Registry are you using? If the NiFi-Registry version is older then 0.5.0, you will encounter this bug. https://issues.apache.org/jira/browse/NIFI-6025 Hope this helps, Matt
... View more
01-08-2020
05:03 PM
2 Kudos
@VijaySankar Is there a full stack trace that follows the error message in the actual nifi-app.log? Is this a multi-node NiFi cluster that perhaps has multiple Nodes installed and running on the same server? If so, when you start the processor only one node is going to be able to bind to the configured port and the other nodes will throw the exception. Since both nodes are on the same host, the processor will appear as though it is working because the node that did bind will receive the requests. It is also possible that port 9099 is not unused on every host in your NiFi cluster. Since the bulletin you are sharing indicates a specific hostname:port, it is possible that port 9099 is already in use by some other process on that one host. But on your other host(s) it is not in use and binds successfully allowing those host(s) to receive request fine. I recommend stopping the HandleHttpRequest processor and then from command line check to see if there is anything still listening on port 9099. If so, you'll need to use another port that is available on all your NiFi nodes. Hope this helps, Matt
... View more