Member since
07-30-2019
3470
Posts
1642
Kudos Received
1018
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 301 | 05-06-2026 09:16 AM | |
| 488 | 05-04-2026 05:20 AM | |
| 355 | 05-01-2026 10:15 AM | |
| 522 | 03-23-2026 05:44 AM | |
| 393 | 02-18-2026 09:59 AM |
01-10-2022
05:58 AM
@Griggsy The "Advanced" UI capability in the UpdateAttribute processor allows you to setup rules to accomplish exactly what you are describing here in your use case. When you right click on the UpdateAttribute processor you will see the "Advanced" button in the lower left corner. From the Advanced UI you can configure conditional rules. 1. Click "+" to the right of "Rules" to create your first rule. 2. Then click "+" to right of "Conditions" to create your boolean NiFi Expression Language (NEL) statement(s). When using multiple statements, all statements must result in true before the corresponding "Action(s)" would be applied. 3. Then click "+" to right of "Actions" to set create one or more actions that are applied if the condition(s) for this rule are true. The thing is that your example or condition does not make sense to me. You are returning the value from a FlowFile attribute named "hostname" and checking if it contains the string "Mickey" in both NEL statements. However, for each you want to return a different output value? Since both would always return either "true" or "false", which output would be desired? Here is example where i check if "Hostname" attribute contains "Mickey" and if that returns true I set FlowFile attribute "hive_database" to "cartoon". I then have a second rule that checks the "Hostname" FlowFile attribute to see if it contains "Bond" and iid that returns true, I set "hive_database" FlowFile" attribute to "movie". The "conditions" and "actions" look similar for the "Bond" rule except "Mickey" is changed to "Bond" and cartoon is set to "movie" instead of "cartoon". So above rules give you an If/then capability. Another feature of using the advanced UI is the "else" capability. Outside the "Advanced" UI set a attribute for "hive_database": If I were to define actions that set the "hive_database" FlowFile attribute and upon evaluating a FlowFile none of the configured rules that set the "hive_database" attribute are applied, the update Attribute processor will set apply the value set outside the advanced UI. If a rule sets the the attribute "hive_database", then value defined outside advanced UI is ignored/not set. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
01-10-2022
05:24 AM
@Elmer What you described here is the use case for the NiFi Parameters capability. Parameters allow you to set configurations values within NiFi components thaT are unique per environment. Parameters can be used in both non-sensitive and sensitive property ( password) fields. Then in each unique environment you set up a parameter context that uses the parameter name, but with the unique value for that specific environment. Then when you port over your flow definition or template, you can configure the NiFi process group in which contains your ported over flow to use the cluster specific parameter context. No need to go to each component and reconfigure them anymore. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt [1]https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Parameters
... View more
12-14-2021
11:32 AM
@Saraali I'd love to help, but I have never done any scripting to convert to excel format. All I can tell you is there is no native processor available in NiFi that does this conversion. I'd recommend raising a new feature request jira in apache for NiFi: https://issues.apache.org/jira/projects/NIFI/issues/ Perhaps others in the community may be looking for the same capability who are willing to contribute code? Thanks, Matt
... View more
12-13-2021
01:02 PM
@armensano Just to make sure we are clear on your ask here. You have a secured NiFi and NiFi-Registry installation that uses user/client certificates to authenticate your user via the mutual TLS handshake? If that is the case, you should not be seeing a NiFi/NiFi-registry login window. Can you share screenshots? What do you see in the nifi-user.log when you attempt to access NiFi? Thanks, Matt
... View more
12-13-2021
12:14 PM
@sunny_de NiFi can only support a single "authorizer". Your authorizers.xml file attached shows that you have two configured: 1. managed-authorizer 2. file-provider <-- (legacy and deprecated in favor of above) Which authorizer is actually being used by NiFi's core is determined by what you have configured in the following property in the nifi.properties file: nifi.security.user.authorizer= So make sure above is set to "managed-authorizer" This provider then references the: file-access-policy-provider This provider is responsible for generating the "authorizations.xml" file which contains all the authorizations created for your users. It will initially seed this file with the minimum authorizations needed for the configured "initial admin identity" string value (Case sensitive), NOTE: The authorizations.xml file is only generated if it does not already exist. If this file already exists and changes made to the file-access-policy-provider will not be reflected in that file. The expectation is once the initial admin and node policies have been seeded, all additional authorization are granted from within the NiFi UI. The "file-access-policy-provider" then references the: file-user-group-provider This provider is designed to generate the users.xml file if it does not exist and seed it with the initial user identities (initial admin and NiFi nodes if clustered). Seeing as how you're able to see the NiFi UI, that tells me that your initial admin was successfully created and given at least "view the user interface" policy permissions. The initial policy setup will only grant polciy permission related to the canvas if a flow.xml.gz was already present when NiFi was secured or started for the first time. In this case, it appears that you did not have a flow.xml.gz and NiFi created one on first startup which happens after the initial admin was created and authorized. You will however have admin policies assigned for your initial admin that will allow you to setup any additional policy you need. I can see from the "operate" panel in the left hand side of the canvas that the following icon is NOT greyed out: This means that your authenticated user is authorized to set policies on the current selected component (in this case the root process group). Click on this icon and grant yourself at a minimum the following policies: 1. view the component 2. modify the component 3. view the data 4. modify the data 5. view provenance After applying above, the icon across the top of the UI should no longer be greyed out since you are now authorized to make modification to the root process group. So now you can drag and drop items to the canvas. When you add new components (processors, input/output ports, funnels, child process groups, etc) to the canvas they will inherit there policies from the parent process group, but you can always select any component and specifically set policies per component. In a multi-team use NiFi, it is common to create a child process group for each team or person and authorized on the desired members to each process group. This prevents user1 from modifying user 2's process group and components within it and vice versa. For more information on setting users, groups, and policies, here is the link to the NiFi documentation that talks about this topic: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#config-users-access-policies If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
12-13-2021
11:27 AM
@Saraali NiFi on contains components for reading XLXS files and converting them to CSV. There are no native components included with NiFi that can convert the other direction. That being said, NiFi offers numerous processors that allow you to execute your own scripts or external commands against the content of a NiFi FlowFile. Here is an example where someone converted from CSV to Excel by passing the FlowFile content to an externally executed python script: https://stackoverflow.com/questions/68937735/how-to-convert-csv-to-excel-using-python-with-pandas-in-apache-nifi If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
12-13-2021
10:57 AM
@Eristocrat @Eristocrat Logging varies by class. Some classes don't produce anything beyond INFO level, while others will produce even TRACE level logging. With TRACE level being the highest level, you will get all possible logging by using it. Do you have a screenshot of the exception you are seeing? At time of exception, did you find anything logged in the NiFi user and/or app logs? Thanks, Matt
... View more
12-09-2021
12:27 PM
@Eristocrat The "nifi.web.https.host" value on each node should be set to the unique hostname assigned to each of your NiFi nodes. Or are you saying that all the three instances of NiFi in your cluster are all installed on the same server/host? You should not have all your NiFi cluster nodes trying to bind to the same host unless they are all installed on sam host which would be extremely unusual and not recommended. is useful when the client to NiFi presents a different hostname than what was specified in the browser URL. For example, you enter https://nifi-proxy:443/nifi and that directs to a proxy that then routes to one of the NiFi cluster host urls https://nifi-node1:443/nifi. What happens here is NiFi receives a request for connecting to nifi-proxy, but nifi-node1 is not nifi-proxy so it fails. That is because it looks like someone made a request to access server nifi-proxy, but that request landed on nifi-node1. So adding nifi-proxy to the nifi.web.proxy.host list tells NiFi to accept request on nifi-node1 that were destined for nifi-proxy. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
12-09-2021
11:55 AM
1 Kudo
@MarcioMarchiori Depending on which NiFi Expression Language (NEL) function you are using, either of the following two NEL statements should work for you: ${path:replaceAll("\\\\",'/')}/${filename} In above you are using the replaceAll function which expects a java regular expression. Since '\" is an escape character and NiFi is a java application, the first \ escapes the second \ and the third \ escapes the forth \. So \\\\ results in a regex of \\ being applied against the value present in the "path" variable. or ${path:replace('\\','/')}/${filename} In above you are using the replace function which does not path the first argument to be evaluated as a java regular expression but rather takes a string literal. And since '\' is an escape character you need \\ which tells java to treat the second \ as the literal string value to search for. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
12-08-2021
10:42 AM
@IAJ Specific to your case we see the following difference: Local Fingerprint (This is fingerprint of flow from node trying to join cluster): s.email.ConsumeIMAPNO_VALUEorg.apache.nifinifi-email-nar1.13.210 sec30 sec1 secWARNfalseTIMER_DRIVENPRIMARY0Mark Messages as Read=truefolder=Inboxhost=outlook.office365.commail.imap.socketFactory.clas Cluster Fingerprint (This is fingerprint of flow currently being used by the cluster): s.email.ConsumeIMAPNO_VALUEorg.apache.nifinifi-email-nar1.13.210 sec30 sec1 secWARNfalseTIMER_DRIVENALL0Mark Messages as Read=truefolder=Inboxhost=outlook.office365.commail.imap.socketFactory.clas If we look closely we see that you have a consumeIMAP processor in your cluster flow configured to execute on "ALL" nodes. On the node trying to connect the same consumeIMAP processor is configured to execute on "Primary" node only. A common scenario where this can happen is if the node was "Disconnected" from the cluster and a user edited the configuration of this processor at that time. Now that node can not rejoin the cluster because of the mismatch. As other have said the flow loaded from the flow.xml.gz in to NiFi JVM Heap memory must match across all nodes. Since all nodes run the same flow.xml.gz, you can copy the this file form any node connected to the cluster to the node failing to connect. NOTE: Understand that by this, you will lose those local changes made while that node was disconnected. Another scenario is the configuration change was made while both nodes were connected, which means that changed was replicated to all nodes in the cluster. If in your 2 node cluster only one node responded to that replicated request that the change was made, the node(s) that did not would get disconnected. Failure to respond to a replication request can happen most often because of server resource issues and/or network issues. Thank you, Matt
... View more