cannot Copy file from Sandbox to windows using Winscp

[#1]

Hello ,

I am new to Sandbox and just started learning it, i have to deploy an application from pycharm to sandbox, i am following the tutorial https://hortonworks.com/tutorial/setting-up-a-spark-development-environment-with-python/ i installed sandbox through VMWare Pro 14, i want to copy the files pyspark and py4j from sandbox that is mentioned in tutorial, i used WINSCP to copy the file through SCP command but i cannot do it, i used the port number of sandbox 8888 and ip address of sandbox 192.168.XX.XXX and userid root and password sandbox password, i am also confused that in root directory of Sandbox there is a folder /usr/ but this folder does not contain any hdp folder as mentioned in tutorial.. Please ur help is highly appreciated

Thanks

[#2]

are you able to start an SSH session on your sandbox (i.e. with Putty)? The default port for SCP should be 22, which is the SSH port. So if you are able to start the shell, but not get WinSCP connected, it must be your WinSCP config.

[#3]

i am not using putty, i am using winscp and add the hortonworks sandbox ip address which is 192.168.xx.xxx with port 22 and its opening but i am unable to run the commands in tutorials https://hortonworks.com/tutorial/setting-up-a-spark-development-environment-with-python/#deploying-t... i am getting the attached error, also my host file configuration is

192.168.xx.xxx localhost sandbox.hortonworks.com sandbox-hdp.hortonworks.com sandbox-hdf.hortonworks.com , Please help me as i am a newbie

[#4]

with WinSCP you should not have to enter the commands, you can copy the files by just dragging them from the right side (which is the remote machine, here the sandbox) to the left which is your local machine. So try to just move the .Main.py file to the correct dir. In the right side just open the dir where you want to copy the files to.

The comand itself:

scp -P 2222 root@sandbox.hortonworks.com:/usr/hdp/current/spark2-client/python/lib/pyspark.zip ~/HelloSpark/

Is supposed to copy the file /usr/hdp/current/spark2-client/python/lib/pyspark.zip from the server sandbox.hortonworks.com into the directory HelloSpark below your homedir on your local machine. To do so with WinSCP go on the right side to /usr/hdp/current/spark2-client/python/lib/ and there select the file pyspark.zip and drag it to the left side.

What might be important, in the tutorial the SSH port is 2222 (and not the default 22 - by parameter -P 2222).

The commands provided in the tutorial are to be entered in the shell, which means you open a shell (like bash) on your local machine and enter the commands.

The commands actually fail, because they are using the name sandbox.hortonworks.com, but you are connecting via an IP address (192.168.47.128), so if you really want, you can try running the commands when you replaced sandbox.hortonworks.com by 192.168.47.128

I guess the target ~/HelloSpark could also be failing if you are running Windows on your local machine.

[#5]

@Harald Berghoff i am dragging down in winscp from my local machine to sandbox, i also have usr file under the root folder but under usr folder there is no hdp file and no spark2-client file, why is it so? that 's why i cannot proceed, if i am using hdp2.6.3 what could be the workaround in this case?

[#6]

it basically means that no hortonworks is installed (as there is no /usr/hdp directory). The sandbox is supposed to provide you with all the installation done. So either you connect to the wrong machine (unlikely as you are able to login with root), or something is wrong with your sandbox. Would you mind to try to download the sandbox again and import it into your tool (i guess you are using virtual box)?

[#7]

I think I just found the explanation on why your installation is not available:

"You must do SSH on port 2222 when you want to connect to the actual docker container where HDP binaries are installed."

I have just read that information here: https://community.hortonworks.com/questions/167327/hadoop-cmd-not-found-error-putty-hortonworks-sand...

This applies to WinSCP as well, try connecting to port 2222, so change it from the default (22).

[#8]

@Harald Berghoff Thank you so much for your help and prompt responses, your last response helps me a lot, i installed putty and all my problems get resolved, Thanks again for keeping in touch

[#9]

@Sana Faraz

From WinSCP just drag and drop from left to right and the error you are getting img3.png is due to the fact that you dont have an entry for the sandbox on your C:\Windows\System32\drivers\etc\hosts please get the ip of your sandbox

# ifconfig 

then use the IP if eg 192.168.0.154 then the host's entry on your windows laptop/desktop should be in the format IP/FQDN/ALIAS eg

192.168.0.154  sandbox.hortonworks.com   sandbox

Now from you windows run the cmd

ping sandbox.hortonworks.com or sandbox

If it response then your connection between the windows and sandbox is okay and proceed with the copy

[#10]

Thanks @

Geoffrey Shelton Okot in this way i can ping to my sandbox , but my other question is i don't have file i.e mentioned in the tutorial /usr/hdp/current/spark2-client/python/lib/pyspark.zip that i need to copy from sandbox to my local machines, how i can get these files. Your help is highly appreciated.