Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cloudera Manager 5.3.1 Quickstart for Docker

Cloudera Manager 5.3.1 Quickstart for Docker

New Contributor

https://registry.hub.docker.com/u/claytantor/docker-centos6-cloudera-5-3-1/

 
OMG this was hard! (still not perfect) It installs without errors on a m3.xlarge instance in AWS. FYI docker host device mapper is broken for centos6, ubuntu 14 is the recommended docker distro for host, guest doesn't matter. This took a ton of time, I am not sure that CM should run inside docker, topic for future discussion.
 
---
 

docker-centos6-cloudera-5-3-1 is an attempt to create a docker equiv to the cloudera manager quickstart. This is a stretch goal though because there are so many components, user roles and networking configurations required to get cloudera manager working that getting it to work inside a docker container is very hard indeed. I tried a ton of the docker CM images that other users have made, most fail miserably. This one at least installs completely.

 

building
docker build -t claytantor/docker-centos6-cloudera-5-3-1:latest .
running
docker run --privileged=true -i -p 49122:22 -p 2181:2181 -p 7180:7180 \
  -p 7182:7182 -p 50010:50010 -p 50075:50075 -p 50020:50020 -p 8020:8020 \
  -p 50070:50070 -p 50090:50090 -p 8032:8032 -p 8030:8030 -p 8031:8031 \
  -p 8033:8033 -p 8088:8088 -p 8040:8040 -p 8042:8042 -p 8041:8041 \
  -p 10020:10020 -p 19888:19888 -p 41370:41370 -p 38319:38319 \
  -p 10000:10000 -p 21050:21050 -p 25000:25000 -p 25010:25010 \
  -p 25020:25020 -p 18080:18080 -p 7077:7077 -p 7078:7078 -p 9000:9000 \
  -p 9001:9001 -p 8888:8888 -t claytantor/docker-centos6-cloudera-5-3-1:latest /bin/bash
configuring

when you run the manager startup you will need to set the directories for the datanodes, namenodes, secondary namenodes etc. because docker's device mapper confuses the manager's mappings. The dockerfile creates the directories and sets the perms to 777 (dubious approach, see TODOs) so that its a little easier during setup but the only way I have found to set these in manually during setup. Here are the dirs to use:

 

Directories
 | Directory                        | Use             |
 | -------------------------------- | --------------- | 
 | /var/cm/datanode1                | Datanode1       | 
 | /var/cm/datanode2                | Datanode2       |
 | /var/cm/datanode3                | Datanode3       |
 | /var/cm/nn                       | Name Node       |
 | /var/cm/snn                      | Secondary Node  |
 | /var/cm/nm                       | NM dir          |
 | /var/cm/impala                   | Impala          |
 | /var/cm/hive                     | Hive data       |
 | /var/cm/cloudera-host-monitor    | Host monitor    |
 | /var/cm/cloudera-service-monitor | Service monitor |
 | /var/cm/sqoop2                   | Sqoop2 data     |
 | /var/cm/zookeeper                | Zookeeper data  |
TODO