@midee  @janis-ax is correct that this is not possible. "ONLY" sensitive attributes can be encrypted/masked.  Only component properties coded to as a sensitive property field would be able to decrypt an encrypted/masked value.  If setting policies to block specific users from being able to view the data on these components, you may want to look at other options for performing these endpoint authentication/authorization actions.  For example, with the NiFi endpoints you could use mutual TLS via certificates instead of token based authentication for accessing the endpoints.  This does not mean you need to stop using or un-configure the token based authentication methods for access to your secured NiFi. If the endpoint you are trying to reach is not NiFi, you may want to see what other options it may offer for authentication that are not token based. Hope this helped, Matt ... View more

@pritster5  The screenshot shared does tell us some the following: We can see on the AzureBlobStorage "0(2)" displayed in the upper right corner.  This tells us that this processor still has two active threads that have been "terminated" manually by a user (Meaning user tried to stop processor but it failed to stop so the user selected terminate).   Terminating threads simply tells NiFi to release the FlowFile associated to that thread back to the inbound connection queue it belongs to and set processor to send anything returned by those still running threads that are marked as terminated to null.  So since we can tell you still have running threads, the question becomes...  "What are these threads doing?" "Are these thread hung?" "Are thee threads just very long running?" I would suggest using executing following several times (1 - 3 minutes apart) to get a series o thread dumps: ./nifi.sh dump <filename fo dump> Then you should inspect those threads to see perhaps what they are waiting on or if they are making any progress (stack changes) between each thread dump.   This will answer your questions above. Maybe connection to AzureBlob Storage is very slow? Maybe Host resources are high and your threads are waiting on CPU? etc... If you found this assisted with your query, please take a moment to login and click "Accept" on solutions that helped you. Thank you, Matt ... View more

@robnew  In addition to what @janis-ax suggested about setting bulletin level to none on the processor to stop WARN and ERROR level log output from that processor from being sent to the bulletin board, you could also suppress the logging of this processor class via the NiFi logback.xml file.  Changing the bulletin level on a processor does not disable logging being produced by your processor code.  That logging will still find it way in to the NiFi logs via the logback.xml even with bulletin level set to known. So though processor you can stop the bulletins and through NiFi's logback.xml you have additional options to send logging produced by your processor class to its own appender or one of the existing appenders already defined in that file. If you found the feedback provided to your query helped, please take a moment to login and click "Accept" on all solutions that helped. Thank you, Matt ... View more

@midee  The invokeHTTP processor will not execute against multiple FlowFiles collectively.  You need to have one FlowFile that contains both your token in a FlowFile attribute and the desired content you are trying to post in the content. The dataflow that obtains the token could be merged with the other dataflow that has yoru content. If you have multiple FlowFiles that will need to use the same token, yo could have to separate dataflows. One that gets the token and rights it to a distributed Map Cache via the PutDistributeMapCache processor. Second dataflwo the gets your content file and fetches the token via the FetchDistributedMapCache processor. Then you have one FlowFile with the token you need in an attribute and the content on the FlowFile to be transferred via InvokeHTTP. If you found this assisted with yoru query, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@LucasMorgado  I don't see any ERROR? Was your old NiFi secured? Is the new NiFi secured? Along with your flow.xml.gz, did you also copy over your users.xml and authorizations.xml files to your new installation? Your screenshot indicates that your user is not authorized to see any of the components (Processors, Process groups, labels, etc), so they are all ghost implementations. Make sure your authorizers.xml is setup just like your old NiFi and that the above mentioned files were also copied over to your new install.  NIFi only reads these files on startup. If you found this addressed yoru question, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@Mahi123    Just to make sure i understand your use case correctly, you want to change the content only of your NiFi FlowFile? You have a NiFi FlowFile with following json content: { "name": "Jon", "title": "king of the nights watch" } and you want to modify it to: { "name": "Jon_<current date>", "title": "king of the nights watch" } I read your statement "I want to change the name of the file" as wanting to edit the filename and not the content of the file itself.  Which is something totally different than above. Assuming you are looking to modify content and not the filename itself, you can do this using the ReplaceText processor after your EvaluateJsonPath processor.   Search Value = ("name": "${name}") Replacement Value = "name": "${name}_${now():format('yyyy-MM-dd-HH-mm-ss')}" Replacement Strategy = Regex Replace Evaluation Mode = Line-by-Line NiFi typically has more than one way to satisfy the use case and there may be better option than the above using the various available record based processors.  You may want to explore those as well. If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@techNerd  Questions for you: 1. Is Computer B consuming all the files LocalFolder and from every sub-directory within c:\users\LocalFolder?  If yes, then your SFTP processor should be configured to Search Recursively. 2. Once consumed by computer B, are the original file being left on Computer A?  If yes, you do NOT want to use the GetSFTP processor as it will refetch the same files over and over again each time it executes since they are not being removed from Computer A.   A typical dataflow here would look like this: The ListSFTP processor would be configured to use as follows: The "Tracking Entities" listing strategy will help when source files may have older timestamps than files previously listed from a different directory.  Using this listing strategy requires you to set a "Entity Tracking State Cache".  There are multiple cache services to choose from.  I simply used the "DistirbutedMapCacheClientService" controller service which gets configured to point at a "DistributedMapCacheServer"controller service I also setup within the same NiFi. The "Remote Path" is set to the top most folder you want to start listing files from ( C:\User\LocalFolder\ ). The "Search Recursively" property is set to "True" so that any random new sub folders added are also searched. The "Success" relationship is then routed via a connection from the ListSFTP to the FetchSFTP processor.   The FlowFiles produced by the ListSFTP processor will have numerous FlowFile Attributes set on them that will be used later to persist your directory structure and to fetch the content via the FetchSFTP processor. You can list the flowfiles on a queue and click the "view details" icon next to nay listed FlowFile to see the attributes currently assigned to that FlowFile. For example:   filename = testfile-32.txt path = /tmp/LocalFolder sftp.remote.host = <SFTP server hostname> sftp.remote.port = 22   Then if you look at attributes for file from an added SubFolder:   filename = testfile-31.txt path = /tmp/LocalFolder/SubFolder sftp.remote.host = <SFTP server hostname> sftp.remote.port = 22     The FetchSFTP processor then uses these attributes set on each FlowFile to fetch the actual content from the source SFTP server: Downstream in your dataflow after you have fetched the content, you can still use the Attributes on each FlowFile when working with yoru FlowFiles. For example using the "path" attribute to dynamically control where a FlowFile will be written on a target system (might be the local NiFi via PutFile processor,  another SFTP server somewhere else, etc.). If you found this addressed your query, please take a moment to login and click "Accept" on all provided solutions that helped. Thank you, Matt ... View more