Created on 01-06-2018 06:10 AM - edited 09-16-2022 05:42 AM
Hi
I am completely new to Sandbox and I am struggling with connection to my local windows PC. All I have done is the following:
1. Downloaded and installed Oracle VirtualBox Manager.
2. Imported applicance - Hortonworks Sandbox.
3. Started my VM and logged into Ambari via my browser with maria_dev.
Here is where I have a problem. How can I get the VM or Hortonworks to talk to my windows PC so that I can transfer files via command line?
I know that I can use the upload via Ambari but I wish to transfer multiple files at once.
What other steps are required? Do I need to set up another user in sandbox?
Any help or advice for a newbie would be greatly appreciated.
Thanks.
Created 01-06-2018 11:11 PM
You will need to do a SSH to the Sandbox first and then run the following commands:
# su - maria_dev # hadoop fs -copyFromLocal /root/sample.txt /user/maria_dev/ (OR) # hdfs dfs -put /root/sample.txt /user/maria_dev/ # hdfs dfs -ls /user/maria_dev/
NOTE: the user "maria_dev" should have read permission atleast on the file "/root/sample.txt" before it tried to read/push it to HDFS.
.
Created 01-06-2018 11:38 AM
you should use a tool like WinSCP to copy files. you will have to configure the host, which is your sandbox (the name or IP should be known, since you used it to connect via browser).It will be possible to use the user maria_dev as well.
If you want to use the shell, you can use a ssh client for Windows, like Putty.
Of course there are other ways as well to allow file exchange (like creating a network share used by Windows and the sandbox), but they will all need some deeper Linux/Windows knowhow, so the simple way is WinSCP. If you like to go with network shares, let me know.
Created 01-06-2018 10:11 PM
@ Harald Berghoff
thank you for your reply.
I have installed WinSCP and I can see 2 drop down folders:
1. / <root>
2. root
Which one is the correct directory? I have moved one file over as an example (called sample.txt) from my PC to the root folder as shown in the attached screen shot.
How can I now move this file over to HDFS (or Sandbox?) using command line with Putty.
I am logged into Putty as shown in the attached screen shot.
Created 01-06-2018 11:11 PM
You will need to do a SSH to the Sandbox first and then run the following commands:
# su - maria_dev # hadoop fs -copyFromLocal /root/sample.txt /user/maria_dev/ (OR) # hdfs dfs -put /root/sample.txt /user/maria_dev/ # hdfs dfs -ls /user/maria_dev/
NOTE: the user "maria_dev" should have read permission atleast on the file "/root/sample.txt" before it tried to read/push it to HDFS.
.
Created 01-06-2018 11:59 PM
Good to know that you were able to move the files to HDFS.
I have successfully moved a file (and now subsequently a folder which was my initial aim) to the maria_dev folder
.
As the original issue (for which the thread was opened) is resolved, hence it will be also great if you can mark this HCC thread as Answered by clicking on the "Accept" Button on the correct answer. That way other HCC users can quickly find the solution when they encounter the same issue.
For new queries/errors it is better to open a separate thread that way we can keep the HCC threads specific to One issue one resolution model.
Created 01-06-2018 11:27 PM
@ Jay Kumar SenSharma
many thanks. I have successfully moved a file (and now subsequently a folder which was my initial aim) to the maria_dev folder.
How can I now join all the files in the folder to one single csv file?
I tried:
hadoop fs -getmerge Folder1 /user/maria_dev/output.csv
But I get the error:
getmerge: Mkdirs failed to create file:/user/maria_dev (exists=false, cwd=file:/home/maria_dev)
I am trying to join all the files in Folder1 folder to a file called "output.csv" in the same folder.
Thanks
Created 01-07-2018 12:01 AM
@ Jay Kumar SenSharma
thank you for letting me know, as I am new to the community. I will mark this as answered and open another question.