Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on 11-29-2024 07:59 AM - edited 11-29-2024 08:01 AM
I'm trying to get a set of file .csv from a directory in a server sftp. My nifi is deployed in a cluster with only one node for nifi. If I try to download a file .csv through sftp> from the same node on which nify is deployed, I can get it without problem. However the processor SFTP in nifi is not able to do it, but I continuosly get the following stacktrace
ERROR org.apache.nifi.processors.standard.FetchSFTP: FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254] Failed to fetch content for StandardFlowFileRecord[uuid=ccf2a3ae-51cc-470c-80e0-6477b045a536,claim=,offset=0,name=file.csv,size=0] from filename /test/nifi on remote host 10.100.201.248:22 due to org.apache.nifi.processor.exception.ProcessException: IOException thrown from FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254]: java.io.IOException: error; routing to comms.failure: org.apache.nifi.processor.exception.ProcessException: IOException thrown from FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254]: java.io.IOException: error
org.apache.nifi.processor.exception.ProcessException: IOException thrown from FetchSFTP[id=1f5b3ddd-2d4c-1889-ffff-ffffa3d58254]: java.io.IOException: error
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2647)
at org.apache.nifi.processors.standard.FetchFileTransfer.onTrigger(FetchFileTransfer.java:251)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1161)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: error
at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1448)
at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1364)
at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
at org.apache.nifi.processors.standard.FetchFileTransfer$1.process(FetchFileTransfer.java:254)
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2628)
... 13 common frames omitted
Can someone helps me? in the FetchSFTP I put the same configuration used to get the file using sftp> thus those are correct. Could it be due to misconfigurations of nifi in the cluster? im using Cloudera
Created 11-29-2024 09:11 AM
@mikecolux Welcome to the Cloudera Community!
To help you get the best possible solution, I have tagged our NiFi experts @MattWho @steven-matison who may be able to assist you further.
Please keep us updated on your post, and we hope you find a satisfactory solution to your query.
Regards,
Diana Torres,Created 12-03-2024 05:24 AM
@mikecolux Can you include a screenshot of how the List/FetchSFTP processors are configured?