NiFi replay at GetFile

2 replies
3855 views
Translate All

I have a GetFile processor followed by a PutSFTP. I simulate SFTP connection issue, and then would like to replay the same flow. However, I can't replay at GetFile as it shows "Cannot replay data from Provenance Event because the event does not contain the required Content Claim". PutSFTP has no data provenance event as well since the SFTP failed. Please advise.

Apache NiFi

MattWhoAnswer

Titanium ⭐️⭐️⭐️

@Adi Sutanto

-

Provenance replay inserts the FlowFile back on to the inbound connection to the processor. Since the GetFile processor does not have any inbound connections, there is no where to insert a FlowFile for replay.

-

As far as PutSFTP processor, you are correct that it generates "SEND" and "DROP" events. If you are auto-terminating the "failure" and/or "reject" relationships, you should see "DROP" events in provenance from which you can perform a "Replay".

If these relationships are being routed via 1 or more connections, then your FlowFile should be sitting in one of those connections allowing you to redirect those FlowFile back to the PutSFTP to try again.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

MattWhoAnswer

Titanium ⭐️⭐️⭐️

@Adi Sutanto

-

Provenance replay inserts the FlowFile back on to the inbound connection to the processor. Since the GetFile processor does not have any inbound connections, there is no where to insert a FlowFile for replay.

-

As far as PutSFTP processor, you are correct that it generates "SEND" and "DROP" events. If you are auto-terminating the "failure" and/or "reject" relationships, you should see "DROP" events in provenance from which you can perform a "Replay".

If these relationships are being routed via 1 or more connections, then your FlowFile should be sitting in one of those connections allowing you to redirect those FlowFile back to the PutSFTP to try again.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

Translate

adisutantoAuthor

Silver ⭐️

Thank you Matt.

Now I understand that we can't replay at GetFile as it has no inbound connection to insert a FlowFile.

As for PutSFTP replay, initially I routed "failure" & "reject" relationships to LogAttribute, that's why I didn't see any "DROP" event in PutSFTP to replay. So, I changed PutSFTP to auto terminate "failure", simulated intermittent connection failure, and then I could replay the "DROP" event which resulted in a successful "SEND" event.

Translate

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.