03-02-2016 01:58 PM
Will there be SFTP built into hadoop-commons anytime soon? I see in HADOOP-5732 that it'll be included in Hadoop 3.8. I'm hoping that I'll be able to use it soon. The reason is that our company uses SFTP to get 3rd party CSV's. It would be great to directly DistCp into HDFS, or, even better yet, directly access the file on the SFTP server using Spark by bringing it into a Dataframe.