Support Questions
Find answers, ask questions, and share your expertise

NiFi Ingest with sftp to solr and banana?

New Contributor

Hi, what processors is best to use with NiFi when I need to pull CSV files via SFTP. Should i first create a GetSFTP then PutHDFS? First whack at this NiFi and not sure how to get started. i would like to do something similar to the solr/banana tweet demo.

Thanks Ron

4 REPLIES 4

Re: NiFi Ingest with sftp to solr and banana?

So I'd have a GetSFTP to pick up the CSV files and then route the flowfiles into both PutSolrContentStream and PutHDFS.

That way you can look at the data both inside and outside of the Solr environment.

Re: NiFi Ingest with sftp to solr and banana?

You might want to look at FetchSFTP. It sounds like you won't be allowed to delete a remote CSV file once downloaded. The FetchSFTP can you an incoming message property to decide which file to download and leave state management out.

Re: NiFi Ingest with sftp to solr and banana?

New Contributor

Thanks! Those are the only 2 processors that I have - GetSFTP and PutSolrContentStream. GetSFTP is complaining about an upstream connection. Not sure why and do I need anything else?

Re: NiFi Ingest with sftp to solr and banana?

Super Collaborator

What is the error message you get from GetSFTP?