@Mohammad Soori Just to make sure I understand correctly... - The TailFile is producing only 20 output FlowFiles; however, all 500 records are included within those 20 FlowFiles. correct? - With a Run Schedule of 0 secs, the processor will be scheduled toe execute and then scheduled to execute again immediately following completion of last execution. During its execution, it will consume all new lines seen since last execution. There is no configuration option that will force this processor to output a separate FlowFile for each line read from the file being tailed. - You could however feed the output FlowFiles to a splitText processor to split each FlowFile in to a separate FlowFile per line. - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum. ... View more

@Benjamin Bouret - The listHDFS processor does not retrieve the actual content of the files. It produces 0 byte FlowFiles that have metadata about the target content. Any hash you produce on these files will not match what the hash produced on the original source ftp server. - If I am not following above correctly, I am not really clear on exactly where you are performing this second hash. How you plan to compare the two hashes. Manually? - NiFi has guaranteed delivery when it writes data to HDFS. If the transfer fails for any reason the FlowFile is routed to failure. - FetchFTP processor also has handling of failures in retrieving the Content: - This check seems like a lot of overhead that should not be necessary. - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum. ... View more

@umang s The following NiFi Expression Language statement will return "true" if a match is found: ${anyDelineatedValue("${temp_array}", ","):contains("${category}")} - Thanks, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer. ... View more

@Tommy - Why even use the "wait" and "notify" processors in this scenario? - All you need to do is feed the "success" relationship form location A PutFile processor to the processor writing to location B. Same file will be written to both locations and will only be written to location B if Location A was successful. - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum. ... View more

@Saikrishna Tarapareddy - You can increase the number of concurrent tasks on each remote port you have established a connection to: Once you have your flow established/connected to your Remote Process Group (RPG), right click on the RPG to bring up teh following context menu: Select "Manage Remote Ports" to open a new UI. You will then be able to edit each Remote Input/Output you have established connection to: You will also want to increase the number concurrent tasks on your remote ports as well. - Thanks, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum. ... View more

@Dan Alan - Anything you can do via the UI, you should be able to accomplish using NiFi's rest-api. https://nifi.apache.org/docs/nifi-docs/rest-api/index.html - The easiest way to learn what specific rest-api call you need to make to accomplish a task is through using the developer tools in your browser. Perform the action via the UI and then via developer tolls save off that action as a curl command. - for example (assume secured NiFi using a login provider): obtain authentication token: token=$(curl -k 'https://<nifi-node>:9091/nifi-api/access/token' --data 'username=<username>&password=<password>’) - Then use that token to stop a specific processor by processor UUID: curl 'https://<nifi-node>:9091/nifi-api/processors/aab961c3-6bcd-18e7-0000-00001d74d4ea' -X PUT -H "Authorization: Bearer $token" -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' --data-binary '{"revision":{"clientId":"248a328f-a133-1cd0-18c1-8997e36ef898","version":1},"component":{"id":"aab961c3-6bcd-18e7-0000-00001d74d4ea","state":"STOPPED"}}' --compressed --insecure - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum. ... View more

@yazeed salem - Please correct me if my below understanding of what you are asking is not correct: - There is no way to click on an existing connection with queued data and select/copy the queued FlowFiles from that connection to another connection. - If you want to join to different connections together, you can use a "funnel". With the processors on both ends of a connection stopped, you will be able to click on that connection and drag the small blue square at destination side of connection to another end-point. This will cause any connections feeding into the funnel to be funneled into a common destination queue. - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum. ... View more