Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.
Labels (1)

Note: HDP 2.5 is used, but can be updated for HDP 2.6 and beyond.

  1. Boot an Amazon Linux instance with at least 16GB of RAM
  2. Execute the following. It will take a while
## install docker
sudo yum update -y
sudo yum install -y docker

## fix docker for importing the large sandbox image
sed -i.backup 's/\(^OPTIONS=.*\)"$/\1 --storage-opt=dm.basesize=20G"/' /etc/sysconfig/docker

## start docker
sudo service docker start

## confirm docker is working
sudo usermod -a -G docker ec2-user
docker info

## download docker image
curl -O http://hortonassets.s3.amazonaws.com/2.5/HDP_2.5_docker.tar.gz

## load docker image
docker load -i HDP_2.5_docker.tar.gz
## confirm image is available
docker images

## get sandbox docker startup script
curl -O https://raw.githubusercontent.com/hortonworks/tutorials/hdp-2.5/tutorials/hortonworks/hortonworks-sa...

## start sandbox
bash start_sandbox.sh

## configure to start at boot
echo "bash /root/start_sandbox.sh" >> /etc/rc.local

## Print the URL for accessing the Sandbox
echo -e "##\nAccess the Sandbox at:\nhttp://$(curl -sS4 icanhazip.com):8888\n##"

Deploying!

7,430 Views
Comments
New Contributor

The very first command fails:

sudo yum update -y

Most references suggest that outbound interfaces are not properly set-up on the instance. How one should set them up in order to get this script to work?

Rising Star

Updated link for HDP 2.6.1 docker image and HDP startup script for Linux:

https://downloads-hortonworks.akamaized.net/sandbox-hdp-2.6.1/HDP_2_6_1_docker_image_28_07_2017_14_4...
https://raw.githubusercontent.com/hortonworks/data-tutorials/master/tutorials/hdp/sandbox-deployment...

It looks like the yum repository with docker is not enabled by default on AWS RHEL EC2 instances. If the yum update works, but you are unable to install docker, try the following to enable the right extras repository:

sudo yum install yum-utils
sudo yum-config-manager --enable rhui-REGION-rhel-server-extras
sudo yum install docker
Rising Star

It also looks like the command to expand the base image from 10GB to 20GB may not work on later versions of docker because the config file is ephemeral. This worked for me, instead (if you run into the dreaded "no space left on device" error.

docker daemon --storage-opt dm.basesize=20G

I used 30G, just to be safe, but the image size for HDP 2.6.1 looks to be about 12.13 GB total, so 20 should do.

Rising Star

One last note: You'll need to update the security rules for your AWS instance. Initially, you have to add a custom TCP rule to allow traffic on port 8888 (usually - Sean's last command will tell you the port). Once you have access, the sandbox portal will give you the ports for all the other services that need to be opened up.

New Contributor

I'm getting consistent "overlay2: Unknown option dm.basesize" in the logfile when I follow your instructions. I've also tried your storage-opt hack below, which produces the same error. I'm using Amazon Linux, and have plenty of storage config'd. Any ideas ?

New Contributor

(Responding to myself) I finally managed to get everything running, despite a very brittle and overly complicated process. See my gist: https://gist.github.com/renodino/5bd8adc661bd2f00a5b662ed0713fd0f

Not applicable

The startup script link is no longer available, and I have searched quite a bit to find it without success. Does anyone have an idea where it has moved to?

Rising Star

The tutorial from which the script was linked has been deprecated on github, but there is a similar set of instructions here:https://docs.hortonworks.com/HDPDocuments/Cloudbreak/Cloudbreak-2.5.1/content/aws-launch/index.html

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎05-19-2017 04:12 PM
Updated by:
 
Contributors
Top Kudoed Authors