Passing ListSFTP parameters on NiFi

Question

Hello, everyone,I'm just starting to get to grips with NiFi and I'm developing a flow that involves taking csv files from an SFTP server and transferring them locally.The task itself seems very simple to me, I found that a viable solution would be to use ListSFTP and FetchSFTP for the purpose. My problem lies in the connection to the SFTP server and in particular in passing the 'Private Key Path' parameter. My private key is stored in a .ppk file on the virtual machine I am using, which uses UNC paths. The moment I pass the UNC path to the processor with the pointing to the .ppk file, the processor returns an alert, telling me that the file does not exist. I can't figure out if the problem is in the way I passed the path to the processor or if it's something else.The path I passed to the processor looks like this:\computer_name\View_UserData$\user_name\Desktop\private_key.ppk In addition, I have set the Port parameter to 22, and the connection also requires a Password.Can anyone give me some tips on how to configure the processor correctly?

MattWho · Accepted Answer

@_fe_20

How exciting that you are diving in to NiFi.
One things that those new to NiFi need to understand is that the dataflows execute independent of the authenticated user who built the dataflow. This means that every component (processor, controller service, reporting task, RPG, etc...) added by whatever user is actually being executed by the NiFi service user and not the user who happens to be logged in to the NiFi UI.

So lets say your NiFi process is owned by a local "nifi" user. This means the ListSFTP and FetchSFTP processors are executed as the nifi user even though you have configured a different user in the processor's configuration. So just like from a terminal windows this would look like:
"sftp -oIdentityFile=/path/to/private/keyfile <username>@<sftpserver>" is being executed by the nifi service user. So when it comes to the private key configured in the processor, it must be owned by the nifi service user.

Now on to issue two. You are using a Putty Private Key (PPK). NiFi does not use putty, so you would need to extract your private key from the ppk file. for example:

puttygen <yourppk>.ppk -O private-openssh -o <your>.pem

Place this private pem key in a directory owned and accessible by the NiFi service user. Make sure permission are not open to group or other since this is a private key.

If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.