Member since
07-30-2019
3436
Posts
1632
Kudos Received
1012
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 132 | 01-27-2026 12:46 PM | |
| 558 | 01-13-2026 11:14 AM | |
| 1199 | 01-09-2026 06:58 AM | |
| 1003 | 12-17-2025 05:55 AM | |
| 494 | 12-17-2025 05:34 AM |
07-28-2021
10:53 AM
1 Kudo
@midee @janis-ax is correct that this is not possible. "ONLY" sensitive attributes can be encrypted/masked. Only component properties coded to as a sensitive property field would be able to decrypt an encrypted/masked value. If setting policies to block specific users from being able to view the data on these components, you may want to look at other options for performing these endpoint authentication/authorization actions. For example, with the NiFi endpoints you could use mutual TLS via certificates instead of token based authentication for accessing the endpoints. This does not mean you need to stop using or un-configure the token based authentication methods for access to your secured NiFi. If the endpoint you are trying to reach is not NiFi, you may want to see what other options it may offer for authentication that are not token based. Hope this helped, Matt
... View more
07-28-2021
10:40 AM
@midee You are using an InvokeHTTP processor to hit the rest-api endpoint of some authentication service to obtain a JWT Token, correct? The invokeHTTP processor gives you only two options for handling the response from the endpoint. Placed it in the content of the FlowFile or in to an attribute of the FlowFile. This means that any user who has been granted authorization to view the data of any component that this FlowFile will traverse can see the attributes and content of this FlowFile. There is no capability for the invokeHTTP to encrypt/mask the endpoint response in either options. Even if there was, this now masked/encrypted data would no longer be usable by downstream components. I suspect your plan is to then use this token to make additional requests to other endpoints later in your flow? Also keep in mind that a token generally has a limited lifespan. You should also have the ability to invalidate the token via another endpoint when you are done with it. So your options here are: 1. Limit user's access so they can not view the data in the dataflows that utilize this token so that they can't see the token in the content or attributes. If you are building the flow, then you know how to get the token so does not matter that you can see the token via the attributes or content. 2. Invalidate the token once done with it. Token can't be used anymore even if a user gained access to it later via looking at the content or attributes of a FlowFile. So you may consider building yoru flow to get a token --> use that token for other actions --> invalidate token (Using NiFi login authentication as a JWT token example, you hit the logout endpoint to invalidate the token on the server side. This means the client token will no longer work even if you have it still). Of course this means you flow needs to obtain a new token every time it runs. Issue here is if your flow utilizes same token for multiple FlowFiles. First FlowFile that hist logout will invalidate token other FlowFiles may try to still use. Note that the default life of a token issued by NiFi is 12 hours. After that the client token can't be used anymore and a new token would need to be obtained anyway (of course this age can be adjusted via the NiFi login provider configuration) If you found this answered your query, please take a moment to login and click "Accept" on the solutions that assisted you. Thank you, Matt
... View more
07-28-2021
10:18 AM
1 Kudo
@pritster5 The screenshot shared does tell us some the following: We can see on the AzureBlobStorage "0(2)" displayed in the upper right corner. This tells us that this processor still has two active threads that have been "terminated" manually by a user (Meaning user tried to stop processor but it failed to stop so the user selected terminate). Terminating threads simply tells NiFi to release the FlowFile associated to that thread back to the inbound connection queue it belongs to and set processor to send anything returned by those still running threads that are marked as terminated to null. So since we can tell you still have running threads, the question becomes... "What are these threads doing?" "Are these thread hung?" "Are thee threads just very long running?" I would suggest using executing following several times (1 - 3 minutes apart) to get a series o thread dumps: ./nifi.sh dump <filename fo dump> Then you should inspect those threads to see perhaps what they are waiting on or if they are making any progress (stack changes) between each thread dump. This will answer your questions above. Maybe connection to AzureBlob Storage is very slow? Maybe Host resources are high and your threads are waiting on CPU? etc... If you found this assisted with your query, please take a moment to login and click "Accept" on solutions that helped you. Thank you, Matt
... View more
07-28-2021
10:07 AM
2 Kudos
@robnew In addition to what @janis-ax suggested about setting bulletin level to none on the processor to stop WARN and ERROR level log output from that processor from being sent to the bulletin board, you could also suppress the logging of this processor class via the NiFi logback.xml file. Changing the bulletin level on a processor does not disable logging being produced by your processor code. That logging will still find it way in to the NiFi logs via the logback.xml even with bulletin level set to known. So though processor you can stop the bulletins and through NiFi's logback.xml you have additional options to send logging produced by your processor class to its own appender or one of the existing appenders already defined in that file. If you found the feedback provided to your query helped, please take a moment to login and click "Accept" on all solutions that helped. Thank you, Matt
... View more
07-19-2021
05:46 AM
@chams221
Have you resolved your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
07-19-2021
05:44 AM
@midee have you resolved your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
07-14-2021
11:22 PM
@MattWho Thanks for your reply and let me know about the correct flow.
... View more
07-13-2021
12:38 PM
@Ash1 Not clear from yoru query how files are getting into or out of your NiFi. Assuming you have already received a set fo FlowFiles for the day in to your NiFi dataflow, the best approach may be to notify if any fails to be written out /transferred at the end of your dataflow. In this manor not only would you know that not all Files transferred, but would know exact what file failed to transfer. There are numerous processors that handle writing out FlowFile content (transfer) to another source or local file system. Those processing components typically have relationships for handling various types of failures. These relationships could be sent through a retry loop via the RetryFlowFile [1] processor back to the same transfer processor that failed. You define in the RetryFlowFile processor how many times you want a FlowFile to traverse this loop. After X number of loop it would get routed out of the loop to your PutEmail [2] processor where you could dynamically set the email content to include attributes from that FlowFile like filename, hostname of NiFi that failed to transfer it, etc... From the PutEmail processor you could send that FlowFile to somewhere else for holding until manual intervention was taken in response to that email. [`1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.RetryFlowFile/index.html [2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.PutEmail/index.html If you found any of the response given here assisted with your query, please take a moment to login and click "Accept" on each of those solutions. Thank you, Matt
... View more
07-13-2021
06:46 AM
@Indhu The more detail you provide the better community members can assist with your query. Can you share an example input and the desired output date string? What format is your source date in? How are you ingesting that data in to NiFi? Thank you, Matt
... View more
07-12-2021
01:31 PM
Hy! Thanks for the answer! My old nifi wasn't secured and didn't have users. In my new installation we have some users, in this error after inserting and configuring acess policies my error was resolved. Thank you!
... View more