@pritster5  The screenshot shared does tell us some the following: We can see on the AzureBlobStorage "0(2)" displayed in the upper right corner.  This tells us that this processor still has two active threads that have been "terminated" manually by a user (Meaning user tried to stop processor but it failed to stop so the user selected terminate).   Terminating threads simply tells NiFi to release the FlowFile associated to that thread back to the inbound connection queue it belongs to and set processor to send anything returned by those still running threads that are marked as terminated to null.  So since we can tell you still have running threads, the question becomes...  "What are these threads doing?" "Are these thread hung?" "Are thee threads just very long running?" I would suggest using executing following several times (1 - 3 minutes apart) to get a series o thread dumps: ./nifi.sh dump <filename fo dump> Then you should inspect those threads to see perhaps what they are waiting on or if they are making any progress (stack changes) between each thread dump.   This will answer your questions above. Maybe connection to AzureBlob Storage is very slow? Maybe Host resources are high and your threads are waiting on CPU? etc... If you found this assisted with your query, please take a moment to login and click "Accept" on solutions that helped you. Thank you, Matt ... View more

@midee  The invokeHTTP processor will not execute against multiple FlowFiles collectively.  You need to have one FlowFile that contains both your token in a FlowFile attribute and the desired content you are trying to post in the content. The dataflow that obtains the token could be merged with the other dataflow that has yoru content. If you have multiple FlowFiles that will need to use the same token, yo could have to separate dataflows. One that gets the token and rights it to a distributed Map Cache via the PutDistributeMapCache processor. Second dataflwo the gets your content file and fetches the token via the FetchDistributedMapCache processor. Then you have one FlowFile with the token you need in an attribute and the content on the FlowFile to be transferred via InvokeHTTP. If you found this assisted with yoru query, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@LucasMorgado  I don't see any ERROR? Was your old NiFi secured? Is the new NiFi secured? Along with your flow.xml.gz, did you also copy over your users.xml and authorizations.xml files to your new installation? Your screenshot indicates that your user is not authorized to see any of the components (Processors, Process groups, labels, etc), so they are all ghost implementations. Make sure your authorizers.xml is setup just like your old NiFi and that the above mentioned files were also copied over to your new install.  NIFi only reads these files on startup. If you found this addressed yoru question, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@Mahi123    Just to make sure i understand your use case correctly, you want to change the content only of your NiFi FlowFile? You have a NiFi FlowFile with following json content: { "name": "Jon", "title": "king of the nights watch" } and you want to modify it to: { "name": "Jon_<current date>", "title": "king of the nights watch" } I read your statement " I want to change the name of the file" as wanting to edit the filename and not the content of the file itself.  Which is something totally different than above. Assuming you are looking to modify content and not the filename itself, you can do this using the ReplaceText processor after your EvaluateJsonPath processor.   Search Value = ("name": "${name}") Replacement Value = "name": "${name}_${now():format('yyyy-MM-dd-HH-mm-ss')}" Replacement Strategy = Regex Replace Evaluation Mode = Line-by-Line NiFi typically has more than one way to satisfy the use case and there may be better option than the above using the various available record based processors.  You may want to explore those as well. If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@techNerd  Questions for you: 1. Is Computer B consuming all the files LocalFolder and from every sub-directory within c:\users\LocalFolder?  If yes, then your SFTP processor should be configured to Search Recursively. 2. Once consumed by computer B, are the original file being left on Computer A?  If yes, you do NOT want to use the GetSFTP processor as it will refetch the same files over and over again each time it executes since they are not being removed from Computer A.   A typical dataflow here would look like this: The ListSFTP processor would be configured to use as follows: The "Tracking Entities" listing strategy will help when source files may have older timestamps than files previously listed from a different directory.  Using this listing strategy requires you to set a "Entity Tracking State Cache".  There are multiple cache services to choose from.  I simply used the "DistirbutedMapCacheClientService" controller service which gets configured to point at a "DistributedMapCacheServer"controller service I also setup within the same NiFi. The "Remote Path" is set to the top most folder you want to start listing files from (  C:\User\LocalFolder\ ). The "Search Recursively" property is set to "True" so that any random new sub folders added are also searched. The "Success" relationship is then routed via a connection from the ListSFTP to the FetchSFTP processor.   The FlowFiles produced by the ListSFTP processor will have numerous FlowFile Attributes set on them that will be used later to persist your directory structure and to fetch the content via the FetchSFTP processor. You can list the flowfiles on a queue and click the "view details" icon next to nay listed FlowFile to see the attributes currently assigned to that FlowFile. For example:   filename = testfile-32.txt path = /tmp/LocalFolder sftp.remote.host = <SFTP server hostname> sftp.remote.port = 22   Then if you look at attributes for file from an added SubFolder:   filename = testfile-31.txt path = /tmp/LocalFolder/SubFolder sftp.remote.host = <SFTP server hostname> sftp.remote.port = 22     The FetchSFTP processor then uses these attributes set on each FlowFile to fetch the actual content from the source SFTP server: Downstream in your dataflow after you have fetched the content, you can still use the Attributes on each FlowFile when working with yoru FlowFiles. For example using the "path" attribute to dynamically control where a FlowFile will be written on a target system (might be the local NiFi via PutFile processor,  another SFTP server somewhere else, etc.). If you found this addressed your query, please take a moment to login and click "Accept" on all provided solutions that helped. Thank you, Matt ... View more

@techNerd  Based on yoru example, you can actually modify your XML using the ReplaceText [1] processor. The processor can be configured with Java regex that would match on the string you want to remove and the replacement value would be set to an empty string. Your regex pattern in this case would need to match on (don't forget the spaces): ( unit = ".*?") The parenthesis mark the java regex capture group. .*? is a non greedy match on any character for zero or more characters until the very next quote.  This allows this to match on any variety of unit strings (  meter, kilogram, and lb) from your example This will result in the following: <particular> <name>Nicholas</name> <height unit = "meter">183</height> <weight unit = "kilogram" unit = "lb">75</weight> </particular> <particular> <name>Debbie</name> <height unit = "meter">163</height> <weight unit = "kilogram" unit = "lb">45</weight> </particular> being converted to: <particular> <name>Nicholas</name> <height>183</height> <weight>75</weight> </particular> <particular> <name>Debbie</name> <height>163</height> <weight>45</weight> </particular>   [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.ReplaceText/index.html If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@ubet  You can run NiFi-Registry on Windows. The exception you are see is related to being unable to load a class that is included with NiFi-Registry. Maybe your download is no good.  Did you try downloading the NiFi-Registry package again? Make sure you purge the work directory that was created by NiFi-Registry when the service attempted to start.  Make sure you did not run out of disk space. You'll also want the product installation directory path and the directory path for the java JDK being used to NOT have spaces in the directory names.  Use " Apache-nifi-1.12.1" instead of "Apache nifi-1.12.1". Reinstall your JDK outside of "Program Files". The above changes and checks should get you up and running successfully. If you found this addressed yoru query, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more

@Arash  The " nifi.remote.input.socket.port" property is used so that the receiving NiFi can support the RAW transport protocol.  This port has nothing to do with where within your dataflows the transferred data is being ingested. The Remote Process Group (RPG) that exists within your MiNiFi dataflow acts as a client.  It will execute a background thread every 30 seconds that connects to the target NiFi URL configured in the RPG that fetches NiFi Site-To-Site (S2S) details from the target NiFi.  These S2S details include details about the target.  This information includes but is not limited to the following: 1. Hostnames of all nodes in target NiFi cluster 2. Remote (RAW) input socket port if configured 3. FlowFile load on each cluster node 4. Remote input and output ports this client is authorized access to. There is no option to configure multiple remote input socket port values.  RPG would not know how to use them even if there was.  In your case each unique MiNiFi should have be pushing to a different Remote input port on the canvas instead of all of them sending to same input port.     Second option is to use a RouteOnAttribute processor after your single Remote input port that routes data based on the "s2s.host" attribute value set on the received FlowFiles.  This attribute is set by the sending RPG on each MiNiFi to the hostname of that MiNiFi.  You could of course also set a unique attribute on each FlowFile via na UpdateAttribute processor on each MiNiFi also before sending through the RPG. If you found this addressed your query, please take a moment to login and click "Accept" on solutions that helped you. Thank you, Matt ... View more

@midee  You could use a routeOnContent [1]processor to accomplish this.  You would create a Java regex that matches only on customfield(s) where there is a string wrapped in quotes.  If found, it routes entire FlowFile to the relationship created using the dynamic property's name. RouteOnContent configuration: I noticed in you example you have two customfield entries that do not have "null" "customfield_10001": "This is required value", "customfield_10002": "", Based on my regex provided above, both of these would match resulting in FlowFile being routed to the "NotNull" relationship. If i were to change the second .*? to .+? , then the customfield that contained only quotes would not match (just in case you only want to route when it is not null and not empty. "customfield_.+?": ".*?", versus "customfield_.+?": ".+?", If you found this addressed your query, please take a moment to login and click "accept" on this solution. Thank you, Matt ... View more

@Leopol  Welcome to NiFi! The ListFile [1] processor is only designed to create a 0 byte NiFi FlowFile (no content is fetched). This created NiFi FlowFile simply has a a bunch of Attributes created on the FlowFile that can be used later to actually retrieve the content via the FetchFile [2] processor.   The combination of these two processors allow NiFi to spread the heavy work across multiple nodes in a cluster when the source of the data may not be cluster friendly (for example a remote disk mounted to all nodes in a NiFi cluster).  The ListFile processor would be configured to execute on "Primary Node" only and its success relationship would be routed via a connection to the FetchFile.  That connection would be configured to load balance the 0 byte FlowFiles produced by ListFile.  Then the FetchSFTP processors executing on all nodes would get the now distributed files and fetch the content.  There are other similar list/fetch combinations. Since you have left the FetchFile processor out of yoru dataflow, you are not passing any content to the UnpackContent processor thus resulting in the exception you are seeing.  In that exception you will see details on the FlowFile trying to be unpacked: StandardFlowFileRecord[uuid=cfe7807c-d6ad-4127-b779-75b2f57c0ba6,claim=,offset=0,name=data.zip,size=0] You'll notice the "size=0" which menas it is 0 bytes which is expected since you have not fetched the content for this file yet.   [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.ListFile/index.html [2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.FetchFile/index.html   If you found this helped with your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt ... View more