FetchSFTP cant retrieve files correctly: java.io.IOException: error; routing to comms.failure

mikecolux — Fri, 29 Nov 2024 16:01:30 GMT

I'm trying to get a set of file .csv from a directory in a server sftp. My nifi is deployed in a cluster with only one node for nifi. If I try to download a file .csv through sftp> from the same node on which nify is deployed, I can get it without problem. However the processor SFTP in nifi is not able to do it, but I continuosly get the following stacktrace

ERROR org.apache.nifi.processors.standard.FetchSFTP: FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254] Failed to fetch content for StandardFlowFileRecord[uuid=ccf2a3ae-51cc-470c-80e0-6477b045a536,claim=,offset=0,name=file.csv,size=0] from filename /test/nifi on remote host 10.100.201.248:22 due to org.apache.nifi.processor.exception.ProcessException: IOException thrown from FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254]: java.io.IOException: error; routing to comms.failure: org.apache.nifi.processor.exception.ProcessException: IOException thrown from FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254]: java.io.IOException: error org.apache.nifi.processor.exception.ProcessException: IOException thrown from FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254]: java.io.IOException: error at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2647) at org.apache.nifi.processors.standard.FetchFileTransfer.onTrigger(FetchFileTransfer.java:251) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1161) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: error at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1448) at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1364) at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35) at org.apache.nifi.processors.standard.FetchFileTransfer$1.process(FetchFileTransfer.java:254) at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2628) ... 13 common frames omitted

Can someone helps me? in the FetchSFTP I put the same configuration used to get the file using sftp> thus those are correct. Could it be due to misconfigurations of nifi in the cluster? im using Cloudera

Re: FetchSFTP cant retrieve files correctly: java.io.IOException: error; routing to comms.failure

DianaTorres — Fri, 29 Nov 2024 17:11:47 GMT

@mikecolux Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @MattWho @steven-matison who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Re: FetchSFTP cant retrieve files correctly: java.io.IOException: error; routing to comms.failure

steven-matison — Tue, 03 Dec 2024 13:24:15 GMT

@mikecolux Can you include a screenshot of how the List/FetchSFTP processors are configured?

question Re: FetchSFTP cant retrieve files correctly: java.io.IOException: error; routing to comms.failure in Support Questions

FetchSFTP cant retrieve files correctly: java.io.IOException: error; routing to comms.failure

Re: FetchSFTP cant retrieve files correctly: java.io.IOException: error; routing to comms.failure

Re: FetchSFTP cant retrieve files correctly: java.io.IOException: error; routing to comms.failure