Member since
07-30-2019
3427
Posts
1632
Kudos Received
1011
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 34 | 01-27-2026 12:46 PM | |
| 470 | 01-13-2026 11:14 AM | |
| 928 | 01-09-2026 06:58 AM | |
| 875 | 12-17-2025 05:55 AM | |
| 936 | 12-15-2025 01:29 PM |
09-17-2024
08:37 AM
1 Kudo
@rizalt There is very little detail in your post. NiFi will run as whatever user is used to start it unless the "run.as" property is set in the NiFi bootstrap.conf file. If the user trying to execute the "./nifi.sh start" command is not the root user and you set the "run.as" property to "root", that user would need sudo permissions in linux to start NiFi as the root user. The "run.as" property is ignored on Windows where the service will always be owned by user that starts it. NOTE: Starting the service as a different user then it was previously started at will not trigger a change in file ownership in NiFi directories. You would need to update file ownership manually be starting as a different issue (this includes all NiFi's repositories). While "root" user has access to all files regardless of owner, issues will exist if no root user launches app and files are owned by another user including root. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
09-17-2024
08:10 AM
@Chetan_mn I loaded up the latest NiFi-2.0.0-M4 (milestone 4 release). Loaded up my flow definition used in my NiFi 1.23 version. All seems to work fine sending headers with mix case and seeing the correct attributes created with those mix case headers on FlowFile generated by HandleHTTPRequest processor. InvokeHTTP: You'll see two custom headers (displayName and outerID) added above as dynamic properties. HandleHTTPRequest processor: When I "List Queue" on the connection containing the "Success" relationship from HandleHTTPRequest processor and "view details" the queued FlowFile, the FlowFile attributes look correct. Are you saying you see different? Try using NiFi 2.0.0-M4 (latest) to see if experience is same. At what point in your dataflow are you checking the validating the FlowFile Attributes. Is your custom script maybe handling them wrong? I am not seeing an issue in the HandleHTTPRequest processor with regards to HTTP Header handling. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
09-10-2024
06:42 AM
2 Kudos
Below is a blog post for a flow i made using the nifi api, which will accept a template (XML), load the template, get template, then follow chain of api calls to get the appropriate flow definiton file (JSON). https://cldr-steven-matison.github.io/blog/dataflow-xml-to-json/ The concepts you need to understand are demonstrated with invokeHttp to execute the sample calls. Check the nifi api docs. You can do these calls with curl, etc., but i chose to do them in nifi. Be mindful that to start using the api from outside of nifi, you need to be following the initial api call for an access token to be used with the rest of the calls.
... View more
09-10-2024
01:23 AM
1 Kudo
Thanks @MattWho , Maybe we tried to skip versions and created this problem. But we could solve it. @gourexe I think the solution that we have applied at that moment, was to migrate the flow to 1.16.3 in a fresh installation. Reconfigure all the secrets and then move on the versions. The lesson that we learned is that we should have a process to test and apply new versions from Nifi, as soon as possible. 🙂
... View more
09-09-2024
02:36 PM
I think you issue may be with using the SingleUserAuthorizer and Single user login provider. These out of the box providers were built so that NiFi could be HTTPS enabled securely out of the box. They are not designed to support clustering, nor are they suitable for production NiFi. You'll want to configure your NiFi cluster to use a production ready authorizer (managed authorizer) and user authentication method other then single user so you can have granular access controls per user/team. Most common is the ldap-provider. The documentation provides examples for authorizer.xml setup: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-based-ldap-authentication https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-based-kerberos-authentication https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#ldap-based-usersgroups-referencing-user-dn ETC.... You cluster is most likely not forming completely due to node to node authentication and authorization issue resulting from using the single user authorizer. In a NiFi cluster the node identities (derived from clientAuth certificates in the Mutual TLS exchange) need to be authorized against some NiFi policies like "Proxy user requests". Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
09-09-2024
01:52 PM
@hegdemahendra The FlowFile connection back pressure thresholds are soft limits. Once one of the configured back pressure thresholds on reached or exceeded, NiFi will not allow the processor feeding that connection to get scheduled to execute again. So in your case no back pressure is being applied, so the ConsumeAzureEventHub processor is being allowed to scheduled to execute. During a single execution it is consuming more events then the threshold settings. What is the batch size set to in your ConsumeAzureEventHub processor? Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
09-09-2024
01:07 PM
@AlokKumar The more detail you provide the better response you will get. In this case, more detail around the content/format of the source file (samples are great) and how it is being or will be processed downstream. A NiFi FlowFile is what is passed from one processor component to the next. There is no direct relationship between what processing was done in one processor versus another. Each Processor will read the FlowFile (consists of FlowFile Attributes/metadata and FlowFile content) and execute its processor code against the FlowFile and output a FlowFile(s) to one or more outbound relationships. So processing against the content of A FlowFile becomes the responsibility of each processor separately. So the questions asked correlate to the answer you'll get. With what you shared, I can only suggest that you remove the first line from the content of the FlowFile so that downstream processor will only have line 2 to process against. This can be accomplished easily with a ReplaceText processor. Simply configure it for "Evaluation Mode" = "line-by-line", "line-by-line Evaluation Mode" = "First-Line", and in "Replacement Value" property "Set empty string" check box. The output FlowFile' content, which is sent to the success relationship, will have everything except its first line. Do you need to preserve line 1? If so, maybe use ExtractText to extract line 1 to a FlowFile attribute you can then use after processing to add back to Content again using ReplaceText. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
09-09-2024
12:08 PM
Thanks for the detailed reply. I added a new value in the nifi.properties and it worked as my installation was brand new, but did not know that the exact value needs to be copied over from the previous installation if we are configuring an old installation. Thanks again.
... View more
09-05-2024
07:55 AM
@Salmidin Please create a new community question with the details of your issue. It seem unrelated to this thread issue. The "Policies" missing from the global menu indicates your NiFi is either using Single User Authorizer (default out of box) or is not setup to be secure. NiFi needs to be configured with a production ready authorizer for the "Polices" and "Users" options to be visible in the NiFi Global menu. Fell free to ping me @MattWho in your new community question. Thanks, Matt
... View more