Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to configure ListSFTP + FetchSFTP

avatar
Explorer

I am trying to collect data with NiFi using ListSFTP+FetchSFTP. But in the configuration ListSFTP works correctly but I have problems with the configuration of FetchSFTP. I tried with ${path}/${filename} still the problem persists,I also used only the data path, but that doesn't work either . the problem also I have a folder of several files.

1 ACCEPTED SOLUTION

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
3 REPLIES 3

avatar
Super Mentor

@Justee 

Sharing the configuration of both your listSFTP and FetchSFTP processor, as well as a listing of the target SFTP directory would be helpful to those who could assist you here. 

The listSFTP processor should be configured with only the base path where all the files you want to list exist  in the "Remote Path" property.  If the files you want to list reside within sub-directories of that base directory, make sure you also se "Search Recursively" to true.

The ListSFTP processor will output a NiFi FlowFile for each file that is listed.  Also those produced FlowFiles will have FlowFile attributes set on them with:

MattWho_0-1620325087873.png

 

So you might want to list those FlowFiles in the NiFi connection queue an inspect what values were set on these properties.  Are they correct?  

These attributes can then be used in the FetchSFTP processor to facilitate the fetching of the content for each FlowFile listed.

Hope this helps,

Matt


Screen Shot 2021-05-06 at 2.18.36 PM.png

avatar
Explorer

Hi@MattWho, I have a folder of files. Here is the configuration I tried to set up but the problem is on remote filename I tried: remote filename = ${/home/Data/}  remote fiename = /home/Data/, but both do not work. 

 

ListSFTP  settingsListSFTP settingsFetchSFTP settingsFetchSFTP settings

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login