How can I connect my local windows PC to Sandbox so that I can move files from my PC to Sandbox and then to HDFS?

[#1]

Hi

I am completely new to Sandbox and I am struggling with connection to my local windows PC. All I have done is the following:

1. Downloaded and installed Oracle VirtualBox Manager.

2. Imported applicance - Hortonworks Sandbox.

3. Started my VM and logged into Ambari via my browser with maria_dev.

Here is where I have a problem. How can I get the VM or Hortonworks to talk to my windows PC so that I can transfer files via command line?

I know that I can use the upload via Ambari but I wish to transfer multiple files at once.

What other steps are required? Do I need to set up another user in sandbox?

Any help or advice for a newbie would be greatly appreciated.

Thanks.

[#2]

you should use a tool like WinSCP to copy files. you will have to configure the host, which is your sandbox (the name or IP should be known, since you used it to connect via browser).It will be possible to use the user maria_dev as well.
If you want to use the shell, you can use a ssh client for Windows, like Putty.

Of course there are other ways as well to allow file exchange (like creating a network share used by Windows and the sandbox), but they will all need some deeper Linux/Windows knowhow, so the simple way is WinSCP. If you like to go with network shares, let me know.

[#3]

@ Harald Berghoff

thank you for your reply.

I have installed WinSCP and I can see 2 drop down folders:

1. / <root>

2. root

Which one is the correct directory? I have moved one file over as an example (called sample.txt) from my PC to the root folder as shown in the attached screen shot.

How can I now move this file over to HDFS (or Sandbox?) using command line with Putty.

I am logged into Putty as shown in the attached screen shot.

pic1.jpgpic2.jpg

[#4 · Accepted solution]

@Matt

You will need to do a SSH to the Sandbox first and then run the following commands:

# su - maria_dev
# hadoop fs -copyFromLocal  /root/sample.txt  /user/maria_dev/
(OR)
# hdfs dfs -put  /root/sample.txt  /user/maria_dev/
# hdfs dfs -ls  /user/maria_dev/

NOTE: the user "maria_dev" should have read permission atleast on the file "/root/sample.txt" before it tried to read/push it to HDFS.

.

[#5]

@Matt

Good to know that you were able to move the files to HDFS.

I have successfully moved a file (and now subsequently a folder which was my initial aim) to the maria_dev folder

.

As the original issue (for which the thread was opened) is resolved, hence it will be also great if you can mark this HCC thread as Answered by clicking on the "Accept" Button on the correct answer. That way other HCC users can quickly find the solution when they encounter the same issue.

For new queries/errors it is better to open a separate thread that way we can keep the HCC threads specific to One issue one resolution model.

[#6]

@ Jay Kumar SenSharma

many thanks. I have successfully moved a file (and now subsequently a folder which was my initial aim) to the maria_dev folder.

How can I now join all the files in the folder to one single csv file?

I tried:

hadoop fs -getmerge Folder1 /user/maria_dev/output.csv

But I get the error:

getmerge: Mkdirs failed to create file:/user/maria_dev (exists=false, cwd=file:/home/maria_dev)

I am trying to join all the files in Folder1 folder to a file called "output.csv" in the same folder.

Thanks

[#7]

@ Jay Kumar SenSharma

thank you for letting me know, as I am new to the community. I will mark this as answered and open another question.