question Am I able to copy a file from my mac (/Users/rev/geolocation.csv) to the HDFS (/tmp/admin/data) via SSH? in Support Questions

Am I able to copy a file from my mac (/Users/rev/geolocation.csv) to the HDFS (/tmp/admin/data) via SSH?

Rebel — Mon, 11 Jan 2016 09:54:13 GMT

Hi, I am interested in doing the HDP Certified Developer (HDPCD) Exam. As such I am doing some background work. Within the link (http://hortonworks.com/training/class/hdp-certified-developer-hdpcd-exam/) there are a set of tasks that I was attempting. The first task ‘Input a local file into HDFS using the Hadoop file system shell’ was one I attempted. I can do this via Ambari by going to the HDFS section and simply uploading the file by pressing the browse button and selecting the data file. However I want to do this via command line. Is there anyway I can do this?

My data is stored on my mac /Users/rev/geolocation.csv. I have created a directory in HDFS located: tmp/admin/data. This is where I will copy the geolocation.csv to.

Via SSH (http://127.0.0.1:4200/) I have tried commands like the one below which works fine.

hadoop fs -ls tmp/admin/data

However, when I type the below I get a no such file or directory message.

hadoop fs -put localfile /Users/rev/geolocation.csv /tmp/admin/data

Am I able to copy a file from my mac (/Users/rev/geolocation.csv) to the HDFS (/tmp/admin/data)?

Re: Am I able to copy a file from my mac (/Users/rev/geolocation.csv) to the HDFS (/tmp/admin/data) via SSH?

aervits — Mon, 19 Aug 2019 12:17:58 GMT

@Revlin Abbi

via SSH, you first have to copy the files to the guest machine, (Sandbox) like so

scp -P 2222 -r file root@127.0.0.1:/home/guest

then when you ssh to the machine, the file will be in /home/guest directory

switch to guest user

su guest

if you do ls on the home directory you will see your file

now you need to upload the file to HDFS

hdfs dfs -put file /user/guest

now you can list the file in hdfs

hdfs dfs -ls /user/guest/

alternatively, you can use the Ambari HDFS Files view to upload directly from your mac to HDFS in Sandbox. The one thing you need to remember is that when you copy data from host to guest you copy to the guest's local filesystem. Loading to HDFS will take another step.

Re: Am I able to copy a file from my mac (/Users/rev/geolocation.csv) to the HDFS (/tmp/admin/data) via SSH?

Rebel — Tue, 12 Jan 2016 00:13:33 GMT

Hi Artem, thank you for help.