https://community.cloudera.com/t5/Support-Questions/sftp-transfer-to-hdfs-in-spark-as-opposed-to-using-a-command/m-p/50949#M48619 Thanks for the response . I did not see the part of it not running on a cluster as I will be using a cluster. I had one more question why would it not work on a cluster does it have something to do with it being distributed like in general? Wed, 15 Feb 2017 14:25:48 GMT gimp077 2017-02-15T14:25:48Z