Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

"What happens in Nifi when attribute of processor FetchSFTP "Connection timeout:30 sec" reaches its max limit i.e exceeds the threshold ?How will the processor FetchSFTP will react ?it will go to fail state ?Same question for attribute Data Timeout?

Solved Go to solution

"What happens in Nifi when attribute of processor FetchSFTP "Connection timeout:30 sec" reaches its max limit i.e exceeds the threshold ?How will the processor FetchSFTP will react ?it will go to fail state ?Same question for attribute Data Timeout?

New Contributor
 
1 ACCEPTED SOLUTION

Accepted Solutions

Re: "What happens in Nifi when attribute of processor FetchSFTP "Connection timeout:30 sec" reaches its max limit i.e exceeds the threshold ?How will the processor FetchSFTP will react ?it will go to fail state ?Same question for attribute Data Timeout?

Master Guru

@amanjain

The FlowFile would be rooted to the failure relationship in both those case. Those FlowFiles would be penalised based on the penalty duration configured on the fetchSFTP processor (default of 30 secs). That FlowFile will not be processed by the processor it is connected to until that penalty has expired. The common scenario her is to have the failure relationship loop back on the fetchSFTP processor so after penalty has expired another attempt will be made to retrieve the data.

Matt

4 REPLIES 4

Re: "What happens in Nifi when attribute of processor FetchSFTP "Connection timeout:30 sec" reaches its max limit i.e exceeds the threshold ?How will the processor FetchSFTP will react ?it will go to fail state ?Same question for attribute Data Timeout?

Master Guru

@amanjain

The FlowFile would be rooted to the failure relationship in both those case. Those FlowFiles would be penalised based on the penalty duration configured on the fetchSFTP processor (default of 30 secs). That FlowFile will not be processed by the processor it is connected to until that penalty has expired. The common scenario her is to have the failure relationship loop back on the fetchSFTP processor so after penalty has expired another attempt will be made to retrieve the data.

Matt

Re: "What happens in Nifi when attribute of processor FetchSFTP "Connection timeout:30 sec" reaches its max limit i.e exceeds the threshold ?How will the processor FetchSFTP will react ?it will go to fail state ?Same question for attribute Data Timeout?

New Contributor

@Matt Thanks Matt for the quick reply

Re: "What happens in Nifi when attribute of processor FetchSFTP "Connection timeout:30 sec" reaches its max limit i.e exceeds the threshold ?How will the processor FetchSFTP will react ?it will go to fail state ?Same question for attribute Data Timeout?

Master Guru

@Aman Jain If you found this information helpful,please accept the answer.

Highlighted

Re: "What happens in Nifi when attribute of processor FetchSFTP "Connection timeout:30 sec" reaches its max limit i.e exceeds the threshold ?How will the processor FetchSFTP will react ?it will go to fail state ?Same question for attribute Data Timeout?

New Contributor

Hi,

I have got some log messages like this when I cannot get a downloaded files (there's no errors flow: comms failed, not found, permission denied) .

I wonder whether this case related to the setting of "Data timeout" and "Connection timeout".

If you think so, please suggest, how can I deal with this problem? What if I put 0 sec to each parameter, does this mean no limit timeout?

Thank you

2018-10-18 08:59:50,661 INFO [Provenance Maintenance Thread-2] o.a.n.p.expiration.FileRemovalAction Removed expired Provenance Event file /nifi-repository-provenance/64054866.prov.gz