Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to copy from local to HDSF

Highlighted

Re: Unable to copy from local to HDSF

Explorer

@ Jay Kumar SenSharma

thank you for your reply.

From Putty I used your command above:

ssh root@127.0.0.1  -p 2222

And received the following error:

ssh: connect to host 127.0.0.1 port 2222: Connection refused

Can you also please explain what you mean by "you will need to SCP your files to Sandbox and then from Sandbox you can put them to HDFS"?

Highlighted

Re: Unable to copy from local to HDSF

Super Mentor

@Matt

I am not sure from putty how you entered that command "ssh root@127.0.0.1 -p 2222"

But the best way to connect to Sandbox is described in the following Tutorial which might be really helpful. https://hortonworks.com/tutorial/learning-the-ropes-of-the-hortonworks-sandbox/#terminal-access

.

Usually if you want to put a file from Windows to Linux host then WinScp kind of UI utilities are the best if you want to do SCP.

Highlighted

Re: Unable to copy from local to HDSF

Super Mentor

@Matt

Using WInSCP once we are able to copy the file to Sandbox then we should be able to run the following commands to push those files to HDFS from Sandbox.

Suppose in Sandbox host the file is present at "/toot/sample.txt" then it can be pushed to HDFS as following:

.

You will need to do a SSH to the Sandbox first and then run the following commands:

# su - maria_dev
# hadoop fs -copyFromLocal  /root/sample.txt  /user/maria_dev/
(OR)
# hdfs dfs -put  /root/sample.txt  /user/maria_dev/
# hdfs dfs -ls  /user/maria_dev/


NOTE: the user "maria_dev" should have read permission atleast on the file "/root/sample.txt" before it tried to read/push it to HDFS.

.


Highlighted

Re: Unable to copy from local to HDSF

Super Mentor

@Matt

If this issue is resolved then it will be also great if you can mark this HCC thread as Answered by clicking on the "Accept" Button on the correct answer. That way other HCC users can quickly find the solution when they encounter the same issue.

Highlighted

Re: Unable to copy from local to HDSF

Explorer

@ Jay Kiumar SenSharma

ok thank you for your help anyway. I appreciate you replying to me even though I have had no success with it.

Re: Unable to copy from local to HDSF

@Matt

Heya. @Jay Kumar SenSharma's is right on point, though it sounds like there's a bit of confusion as to what Hadoop's "copyFromLocal" command does and what it expects.

You run the Hadoop command with the option "copyFromLocal" from within the sandbox. This command will look on the Sandbox's local filesystem and copy the files you specified into the Hadoop filesystem.

I can definitely see how this may be a bit confusing - it sort of sounds like the command would pull in files from your machine into HDFS on the sandbox. Remember, though, that the sandbox exists on a virtual machine that by default is separate from your own and can't access your filesystem.

So you can definitely use that command, but you'll first have to move your local files into the sandbox. There are a bunch of ways to do that in bulk, check out Jay's post regarding SCP.

Hope that clarifies a bit, and good luck! :)

Highlighted

Re: Unable to copy from local to HDSF

Explorer

@Matt,

Hope you were successful in copying your files to HDFS.

For any other "Getting Started Questions" you might have please check out the Learning the Ropes Hortonworks Tutorial.

Don't have an account?
Coming from Hortonworks? Activate your account here