Member since
09-19-2022
4
Posts
0
Kudos Received
0
Solutions
04-09-2023
10:00 PM
@Shelton Please note that the directory on edge server to which I am trying to copy the file is already present. So ideally it should not try to perform mkdir operation. as I mentioned in my first post, it is looking for the directory in cwd of the node where the code is being executed and since it is not able to find it , it is trying to create one. So basically it should look for the directory in edge server instead of the directory mentioned in the cwd of the error posted.
... View more
09-19-2022
05:57 AM
@Sanchari NiFi FlowFiles reside in connection between NiFi component processors. When a processor gets a thread to execute, it takes the highest priority FlowFile from an inbound connection queue and executes the processor code utilizing that FlowFiles metadata/attributes and content (if processor needs content). The FlowFile is not transferred to a processors outbound connection(s) until execution is complete. When NiFi is shutdown gracefully (meaning a user has initiated a shutdown), NiFi stops scheduling future component execution. NiFi then gives existing executing threads a grace period to complete their thread execution. At the end of that grace period, any still running threads are killed with the JVM. Since FlowFiles do not transfer to an outbound connection until code execution has completed, and FlowFile that was owned by a thread at the time the thread was killed still remains on the inbound connection. When NiFi is started again and the dataflows started, the file processing will start over when the processor executes again and executes against the highest priority FlowFile in the connection. Above being said, NiFi will favor data duplication over data loss every time. It is possible in a small window of time that processor executes and part of that execution is let's say to write a file to a remote server. NiFi may for example ack the completion of that transfer to the remote system and NiFi JVM was killed before internally it received ack back from target server. So the FlowFile would end up being processed again resulting potentially data duplication on the target server. These are rare race conditions, but possible. A restart is nothing more than a standard shutdown followed by a start. The same behavior exists in the shutdown process as described above when a restart is performed. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more