Member since
01-26-2017
21
Posts
3
Kudos Received
0
Solutions
03-14-2017
06:58 PM
@vshukla here are the configs for my machines Azure VM (D12 v2 config) is 4 cores, 28 GB RAM, 200GB HDD; VMWare sandbox has 16GB RAM, 8 cores and 1 TB HDDspace. Will this be not enough for Sandbox? The size of file i am trying to upload - 1 MB. I am able to load 5 MB txt file just fine. Only for CSVs it cribs. I will try out the HDC. Thanks for the link!
... View more
03-13-2017
04:36 AM
@vshukla , i did some more digging and found that whenever i run the commands, Yarn-Memory in Ambari dashboard goes upto 90%. What could be wrong?
... View more
03-08-2017
10:04 PM
@vshukla This is the sandbox from hortonworks. so i suppose it is standalone mode; but how do i verify that? Spark-shell - just launch Putty, ssh'd into root, issued pyspark to get to Spark, and issued command (rdd = sc.textFile(csv)) - Works like a charm Zeppelin - exactly the same - used %pyspark interpreter . - doesnt work My Azure VM (D12 v2) config is 4 cores, 28 GB RAM, 200GB HDD; My local VMWare sandbox has 16GB RAM, 8 cores and 1 TB HDDspace. Will this be not enough for Zeppelin?
... View more
03-08-2017
06:27 PM
One more info : I could the same things on txt file, just now csv's. Any pointers?
... View more
03-08-2017
06:26 PM
1 Kudo
@glupu this is exactly what i did, re-imported a new sandbox (and deleted the previous one). The one thing i lost due to that is missed all my Zeppelin notebooks (should have take a bakcup of them).
... View more
03-08-2017
07:39 AM
I am trying to load a csv file into a RDD using textFile function in Zeppelin and then do a take(10). But the take does not produce any result in Zepplin while the same commands outputs rows in SSH (shell) I have attached the file, my zepplin notebook and some screenshots. Can you please suggest me how to resolve this error? (HortonWorks Sandbox HDP2.5 on Microsoft Azure) data : dodgers.zip Zeppelin pras-playground.zip
... View more
03-08-2017
02:34 AM
@Michael Young , I am using imported VMWare Sandbox (not native Docker Sandbox). It was working fine couple of days ago, but failed since I did a restart of my host machine (windows 10).
... View more
03-07-2017
10:50 PM
1 Kudo
Hello all, I am unable to start the sandbox docker container on my VMWare (HDP Sandbox 2.5). The sandbox image and docker service are fine, but iimage is not starting. This is the error message i get: Error response from daemon: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\rootfs_linux.go:89:
... View more
03-02-2017
07:51 AM
Yes this did it 🙂 I tried the sudo su earlier too, but i dint work. I restarted the docker service and did a sudo su , it all worked fine. guess there was some issue with docker service. For anyone who wants to try, i did this After u logged on to the VM via Putty (if ur host OS is Windows) , dont ssh into root@sandbox yet. Stay on the linux VM and do these services docker restart sudo su docker ps
... View more
03-02-2017
07:48 AM
Hello @pdarvasi , the docker is installed as part of the Azure Sandbox. I can see the docker service running. And i was also able to run the docker cp command once i did "sudo su"
... View more
03-01-2017
07:43 AM
Thanks for the response. This dint work in Azure. I get this error: I was able to ssh into port 2222 though, in Azure. Am i missing something? However, this worked in my VMWare workstation.
... View more
02-28-2017
08:01 AM
I was successfuly able to copy a file to Azure from Windows using Putty Copy (pscp). I used the following command pscp - P 22 <file from local folder> tmp\ I could see that file in the sandbox host. Now when i tried to "put" the file from sandbox host to the sandbox itself (docker container), i am not able to do so (throws network not accessible error). The command below hadoop fs -ls -put \tmp\onecsvfile.csv \tmp\ hadoop fs -ls -put \tmp\onecsvfile.csv root@localhost:2222\\tmp\ Nothing works 😞 Can you tell me how can i move the file from sandbox host to the sandbox (Sandbox 2.5 hosted in Azure)?
... View more
01-31-2017
02:45 AM
In essence, how do i query/distinguish local Sandbox file system and hdfs file system? I put the following queries in PuTTy, but it doesnt return anything hadoop fs -ls /tmp <--doenst show the file hdfs dfs -ls /tmp <-- shows the file Am i missing something?
... View more
01-31-2017
02:42 AM
Thank you qureshi. This solved my issue. Is there any documentation on the HortonWorks Sandbox File Sytem? I understand hadoop/hdfs file system are differenct from your response, but is there a clearer documentation on how to navigate around the file system within the Sandbox?
... View more
01-30-2017
11:57 PM
Hello.. I used PuTTy's pscp command to copy a file (Windows 10) to my Sandbox (HDP 2.5, hosted on VMWare Workstation 12). The copy looked sucessfull But i cannot find the file anywhere in the system - i used the command line (PuTTy) as well as Ambari Web interfac; couldnt find it at all. Praskutti is a folder i created in /user/ and in the /root , but cant find the file in neither. Can you guys help me where//how to find the files?
... View more
Labels:
01-27-2017
05:44 PM
1 Kudo
Hi Brandon Actually, i imported the VMWare image and everything was successful (the import process). What I did not realize was that the image in itself is a VM (docker-contained). So when i tried putty into root@sandbox, it was in fact talking to the docker container and not the HDP sandbox. Essentially, we have 2 VMs after the import is complete - VMware creates a CentOS VM and Docker VM inside it to host the actual HDP Sandbox Once i figured out that i have to ssh root@<sandbox IP shown right after the import process> -p2222 , i was able to follow the tutorials. The bold port number is the key here - to access the sandbox within the docker image via PuTTy. Hope this helps for newbies like me. Also if the HortonWorks team could update their documentation to cleary explain this, it would be helpufl. Thanks for your response again!
... View more
01-27-2017
08:52 AM
Hi John, Thanks for the guidance. I in fact added 8888/8080/9995(for Zepellin) and the likes during my intial attempt (a week ago). But it did not allow me to login. But today, i rebuild the VM from scratch and added them again. I am able to login and I am also able to update ambari password. Thanks once again fro your assistance. <Marking your reply as answer>
... View more
01-26-2017
10:41 PM
I created a VM in Azure (VM spec: D12 v2) with HortonWorks Sandbox 2.4 following the guide here: http://hortonworks.com/hadoop-tutorial/deploying-hortonworks-sandbox-on-microsoft-azure/ On step 13, when i tried to connect using http://<public ip from azure>:<8888/8080/80/22/150> , it doesnt connect at all. i tried all major browsers, from home network, work network and mobile network. doesnt help I checked Azure Network Security group settings and i saw an entry for inbound rules to allow SSH. I also created a new Inbound rule to allow HTTP over port 150, but that doesnt work either. allow-http.png network-security-group.png azure-vm-size.png error.png What am i missing here? Any particular configuration i have to do? Side note: PuTTy doesnt connect to the machine as well. I have been banging my head against the wall for a week with this.
... View more
Labels:
01-26-2017
01:13 AM
Does Hortonrworks HDP Sandbox 2.5 support VMWare Workstation 12 Pro (host OS - Windows 10/7 ENT) ?
... View more