Hi, what processors is best to use with NiFi when I need to pull CSV files via SFTP. Should i first create a GetSFTP then PutHDFS? First whack at this NiFi and not sure how to get started. i would like to do something similar to the solr/banana tweet demo.
So I'd have a GetSFTP to pick up the CSV files and then route the flowfiles into both PutSolrContentStream and PutHDFS.
That way you can look at the data both inside and outside of the Solr environment.
You might want to look at FetchSFTP. It sounds like you won't be allowed to delete a remote CSV file once downloaded. The FetchSFTP can you an incoming message property to decide which file to download and leave state management out.
Thanks! Those are the only 2 processors that I have - GetSFTP and PutSolrContentStream. GetSFTP is complaining about an upstream connection. Not sure why and do I need anything else?