HDP2.3-Pig & Hive Rev6 VM for Self Paced Learning

[#1]

Hi:

I am subscribed to the Self Paced Learning by Hortonworks.

One of the modules there is HDP Developer: Apache Pig and Hive Rev 6. For that I downloaded the relevant VM from this link: https://s3.amazonaws.com/vm-us-east/data_analyst/HDP2.3-Pig-Hive-Rev6.zip

I could successfully start the VM using VMware Workstation 12 Player. But I can't see the Hadoop cluster starting. See the screenshot below:

Waited for a long time with no luck.

On the other hand, I could successfully use the Hortonworks Sandbox HDP v2.4 without any trouble.

Any help will be much appreciated.

Thanks and regards,

Anindya

[#2 · Accepted solution]

For this Training VM, there is a hidden folder named .sys in the directory you are in above which contains a recreate_sandbox.sh script that can be used to recreate the Docker instance needed and get everything operational again. You can run it as shown below.

root@ubuntu:~# cd .sys
root@ubuntu:~/.sys# pwd
/root/.sys
root@ubuntu:~/.sys# ./recreate_sandbox.sh
creating sandbox....
docker stop/waiting
docker start/running, process 5857
          inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
sandbox started at 172.17.0.2
root@ubuntu:~/.sys# date
Tue Sep 20 22:15:10 EDT 2016
root@ubuntu:~/.sys# date
Tue Sep 20 22:19:51 EDT 2016
root@ubuntu:~/.sys# ssh sandbox
The authenticity of host 'sandbox (172.17.0.2)' can't be established.
RSA key fingerprint is 2e:0c:53:b1:d4:06:7d:ab:bd:79:f9:17:08:f2:8a:4b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'sandbox,172.17.0.2' (RSA) to the list of known hosts.
Last login: Sun Dec 20 14:06:45 2015 from ip-172-17-0-1.ec2.internal
[root@sandbox ~]# hdfs dfs -ls /
Found 7 items
drwxrwxrwx   - yarn   hadoop          0 2015-10-15 09:45 /app-logs
drwxr-xr-x   - hdfs   hdfs            0 2015-10-15 09:45 /apps
drwxr-xr-x   - hdfs   hdfs            0 2015-10-15 09:44 /hdp
drwxr-xr-x   - mapred hdfs            0 2015-10-15 09:44 /mapred
drwxrwxrwx   - mapred hadoop          0 2015-10-15 09:44 /mr-history
drwxrwxrwx   - hdfs   hdfs            0 2015-10-15 09:46 /tmp
drwxr-xr-x   - hdfs   hdfs            0 2015-10-20 14:31 /user
[root@sandbox ~]#

NOTE: If it fails again in the future, instead of recreating everything, try the restart_sandbox.sh script instead.

[#3]

@Lester Martin, thanks for the info. I did try this approach. However got a funny message when I tried to run the recreate script. See the screenshot below.

Look at the 2 error messages right after creating sandbox....

Also when I tried to issue the hdfs dfs -ls / command, I got the error below.

Connection refused at Sandbox 172.17.0.1

Do I need to wait before I try anything? Any help will be much appreciated. Regards,

Anindya

[#4]

That "no such id: sandbox" is the concerning error to me. I hate to ask, but could you please download the zipped VM again and start all over. I'd like to attach the latest setup guide as well, but HCC has a file size limit that is preventing me from doing that. If it doesn't work this next time, please send an email to training-support@hortonworks.com (you can reference this HCC post, too) which will create an more easy to track internal support case for our Training DevOps team to further help you. We could also attach the latest setup guide that way if needed.

[#5]

@Lester Martin, thanks for your response. So does it mean the online self paced learning doesn't have the latest setup guide? If yes, I believe it is better to update that instead of connecting to individual subscriber. However in the mean time I will drop an email to training-support@hortonworks.com, referencing this post. Also FYI, after waiting may be 10 more minutes I could successfully run the hdfs dfs -ls / command and it listed all those 7 items. Does it mean I can now use the VM or do you still prefer me to download the VM again and start from the scratch? Regards,

Anindya

[#6]

The self-paced library's setup guide is current; I was using that as a ruse to help me move this concern over to our internal tracking system. 😉 Yes, if "hdfs dfs -ls /" is responding, then by all means march forward. If at some point it stops working (i.e. these VM's don't really like to be stopped & started) then please try the restart_sandbox.sh script mentioned earlier with the recreate_sandbox.sh as a "nuclear option". Good luck!

[#7]

@Lester Martin, thanks a lot. I got what you said : '(i.e. these VM's don't really like to be stopped & started)'.

So instead of powering the VM off, I am suspending it and then resuming the same when I need to use it. Cheers!

Anindya

[#8]

@Lester Martin - Thank you for the solution. This resolved my issue.

[#9]

Thanks a lot @Lester Martin!!

finally i am able to get my sandbox up and running after a whole day 🙂

[#10]

@Anindya Chattopadhyay

Can you please try

ssh root@sandbox -p 2222