Pre-requisites:
1. Install Docker in all nodemanager hosts and configure:
It is recommended to install the version of Docker that is provided by your Operating System vendor. The Docker package has been known by several names; docker-engine, docker, and docker-ce.
yum install docker
If having issues installing docker, following document can be followed:
https://docs.docker.com/install/
Edit ‘/etc/docker/daemon.json’ and add the following options:
{
"live-restore" : true,
"debug" : true,
"dns": ["<YARN registry dns ip addr>"]
}If not using HTTPS, Configure each of the cluster hosts to skip HTTPS checks by adding following line in ‘/etc/docker/daemon.json’
"insecure-registries": ["<docker registry server>:5000"]
2. Create a private local docker registry(Optional)
2.1- Designate a server in the cluster for use by the Docker registry. Minimal resources are required, but sufficient disk space is needed to store the images and metadata. Docker must be installed and running.
2.2 Start the registry
docker run -d -p 5000:5000 --restart=always --name registry registry:2
2.3 (Optional)By default, data will only be persisted within the container. If you would like to persist the data on the host, you can customize the bind mounts using the -v option:
docker run -d -p 5000:5000 --restart=always -v /host_registry_path:/var/lib/registry --name registry registry:2
3. Configure YARN to run Docker containers:
Our cluster is now ready to run Dockerized applications.
Dockerize HBASE:
1. Create a yum repo file "hdp.repo" that contains HDP-3.1.0.0 and HDP-UTILS-1.1.0.22 repositories:
[HDP-3.1.0.0] name=HDP Version - HDP-3.1.0.0 baseurl=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0 gpgcheck=1 gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [HDP-UTILS-1.1.0.22] name=HDP-UTILS Version - HDP-UTILS-1.1.0.22 baseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7 gpgcheck=1 gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
2. Create the Dockerfile:
FROM centos:7 ENV JAVA_HOME /usr/lib/jvm/jre-1.8.0-openjdk COPY hdp.repo /etc/yum.repos.d/ RUN yum updateinfo && yum install -y sudo java-1.8.0-openjdk-devel hbase phoenix hadoop-yarn hadoop-mapreduce && yum clean all
3. Build the image:
docker build -t hbase .
4. Tag the image and push it to the docker local registry:
Tag the image as “<docker registry server>:5000/hbase_local”. This creates an additional tag for the existing image. When the first part of the tag is a hostname and port, Docker interprets this as the location of a registry
docker tag hbase <docker registry server>:5000/hbase_local docker push <docker registry server>:5000/hbase_local
Now that our hbase image is created we will create a Yarn Service configuration file (Yarnfile) with all the details of our service
Deployment:
1. Copy core-site.xml, hdfs-site.xml to user dir in HDFS:
su - ambari-qa
hdfs dfs -copyFromLocal /etc/hadoop/conf/core-site.xml .
hdfs dfs -copyFromLocal /etc/hadoop/conf/hdfs-site.xml .2. Create YarnFile (hbase.json):
{
"name": "hbase",
"lifetime": "10800",
"version": "2.0.2.3.1.0.0",
"artifact": {
"id": "<docker registry server>:5000/hbase_local",
"type": "DOCKER"
},
"configuration": {
"env": {
"HBASE_LOG_DIR": "var/log/hbase",
"HADOOP_HOME": "/usr/hdp/3.1.0.0-78/hadoop"
},
"properties": {
"docker.network": "host"
},
"files": [
{
"type": "TEMPLATE",
"dest_file": "/etc/hadoop/conf/core-site.xml",
"src_file": "core-site.xml"
},
{
"type": "TEMPLATE",
"dest_file": "/etc/hadoop/conf/hdfs-site.xml",
"src_file": "hdfs-site.xml"
},
{
"type": "XML",
"dest_file": "/etc/hbase/conf/hbase-site.xml",
"properties": {
"hbase.cluster.distributed": "true",
"hbase.zookeeper.quorum": "${CLUSTER_ZK_QUORUM}",
"hbase.rootdir": "${SERVICE_HDFS_DIR}/hbase",
"zookeeper.znode.parent": "${SERVICE_ZK_PATH}",
"hbase.master.hostname": "hbasemaster-0.${SERVICE_NAME}.${USER}.${DOMAIN}",
"hbase.master.info.port": "16010"
}
}
]
},
"components": [
{
"name": "hbasemaster",
"number_of_containers": 1,
"launch_command": "sleep 15; /usr/hdp/current/hbase-master/bin/hbase master start",
"resource": {
"cpus": 1,
"memory": "1024"
},
"readiness_check": {
"type": "HTTP",
"properties": {
"url": "http://${THIS_HOST}:16010/master-status"
}
},
"configuration": {
"env": {
"HBASE_MASTER_OPTS": "-Xmx1024m -Xms512m"
}
}
},
{
"name": "regionserver",
"number_of_containers": 3,
"launch_command": "sleep 15; /usr/hdp/current/hbase-regionserver/bin/hbase regionserver start",
"resource": {
"cpus": 1,
"memory": "512"
},
"configuration": {
"files": [
{
"type": "XML",
"dest_file": "/etc/hbase/conf/hbase-site.xml",
"properties": {
"hbase.cluster.distributed": "true",
"hbase.zookeeper.quorum": "${CLUSTER_ZK_QUORUM}",
"hbase.rootdir": "${SERVICE_HDFS_DIR}/hbase",
"zookeeper.znode.parent": "${SERVICE_ZK_PATH}",
"hbase.master.hostname": "hbasemaster-0.${SERVICE_NAME}.${USER}.${DOMAIN}",
"hbase.master.info.port": "16010",
"hbase.regionserver.info.port": "16020",
"hbase.regionserver.port": "16030",
"hbase.regionserver.hostname": "${COMPONENT_INSTANCE_NAME}.${SERVICE_NAME}.${USER}.${DOMAIN}"
}
}
],
"env": {
"HBASE_REGIONSERVER_OPTS": "-XX:CMSInitiatingOccupancyFraction=70 -Xmx512m -Xms256m"
}
}
},
{
"name": "hbaseclient",
"number_of_containers": 1,
"launch_command": "sleep infinity",
"resource": {
"cpus": 1,
"memory": "512"
}
}
],
"quicklinks": {
"HBase Master Status UI": "http://hbasemaster-0.${SERVICE_NAME}.${USER}.${DOMAIN}:16010/master-status"
}
}3. Deploy application using the YARN Services API:
yarn app -launch hbase hbase.json
4. Go to "Services" in YARN RM UI and select hbase:
We have 1 hbase master, 3 regionservers and 1 hbaseclient components running.
5. We can also use the Yarn Service REST API to get the state of our hbase service:
curl -X GET 'http://<RM host>:8088/app/v1/services/hbase?user.name=ambari-qa' | python -m json.tool
{
"artifact": {
"id": "<docker registry server>:5000/hbase_local",
"type": "DOCKER"
},
"components": [
{
"artifact": {
"id": "<docker registry server>:5000/hbase_local",
"type": "DOCKER"
},
"configuration": {
"env": {
"HADOOP_HOME": "/usr/hdp/3.1.0.0-78/hadoop",
"HBASE_LOG_DIR": "var/log/hbase",
"HBASE_MASTER_OPTS": "-Xmx1024m -Xms512m"
},
"files": [
{
"dest_file": "/etc/hbase/conf/hbase-site.xml",
"properties": {
"hbase.cluster.distributed": "true",
"hbase.master.hostname": "hbasemaster-0.${SERVICE_NAME}.${USER}.${DOMAIN}",
"hbase.master.info.port": "16010",
"hbase.rootdir": "${SERVICE_HDFS_DIR}/hbase",
"hbase.zookeeper.quorum": "${CLUSTER_ZK_QUORUM}",
"zookeeper.znode.parent": "${SERVICE_ZK_PATH}"
},
"type": "XML"
},
{
"dest_file": "/etc/hadoop/conf/core-site.xml",
"properties": {},
"src_file": "core-site.xml",
"type": "TEMPLATE"
},
{
"dest_file": "/etc/hadoop/conf/hdfs-site.xml",
"properties": {},
"src_file": "hdfs-site.xml",
"type": "TEMPLATE"
}
],
"properties": {
"docker.network": "host"
}
},
"containers": [
{
"bare_host": "pandrade-2.openstacklocal.com",
"component_instance_name": "hbasemaster-0",
"hostname": "hbasemaster-0.hbase.ambari-qa.OPENSTACKLOCAL.COM",
"id": "container_e21_1553187523351_0006_01_000002",
"ip": "172.26.81.14",
"launch_time": 1553266732354,
"state": "READY"
}
],
"dependencies": [],
"launch_command": "sleep 15; /usr/hdp/current/hbase-master/bin/hbase master start",
"name": "hbasemaster",
"number_of_containers": 1,
"quicklinks": [],
"readiness_check": {
"properties": {
"url": "http://${THIS_HOST}:16010/master-status"
},
"type": "HTTP"
},
"resource": {
"additional": {},
"cpus": 1,
"memory": "1024"
},
"restart_policy": "ALWAYS",
"run_privileged_container": false,
"state": "STABLE"
},
{
"artifact": {
"id": "pandrade-4:5000/hbase_local",
"type": "DOCKER"
},
"configuration": {
"env": {
"HADOOP_HOME": "/usr/hdp/3.1.0.0-78/hadoop",
"HBASE_LOG_DIR": "var/log/hbase",
"HBASE_REGIONSERVER_OPTS": "-XX:CMSInitiatingOccupancyFraction=70 -Xmx512m -Xms256m"
},
"files": [
{
"dest_file": "/etc/hbase/conf/hbase-site.xml",
"properties": {
"hbase.cluster.distributed": "true",
"hbase.master.hostname": "hbasemaster-0.${SERVICE_NAME}.${USER}.${DOMAIN}",
"hbase.master.info.port": "16010",
"hbase.regionserver.hostname": "${COMPONENT_INSTANCE_NAME}.${SERVICE_NAME}.${USER}.${DOMAIN}",
"hbase.regionserver.info.port": "16020",
"hbase.regionserver.port": "16030",
"hbase.rootdir": "${SERVICE_HDFS_DIR}/hbase",
"hbase.zookeeper.quorum": "${CLUSTER_ZK_QUORUM}",
"zookeeper.znode.parent": "${SERVICE_ZK_PATH}"
},
"type": "XML"
},
{
"dest_file": "/etc/hadoop/conf/core-site.xml",
"properties": {},
"src_file": "core-site.xml",
"type": "TEMPLATE"
},
{
"dest_file": "/etc/hadoop/conf/hdfs-site.xml",
"properties": {},
"src_file": "hdfs-site.xml",
"type": "TEMPLATE"
}
],
"properties": {
"docker.network": "host"
}
},
"containers": [
{
"bare_host": "pandrade-2.openstacklocal.com",
"component_instance_name": "regionserver-2",
"hostname": "regionserver-2.hbase.ambari-qa.OPENSTACKLOCAL.COM",
"id": "container_e21_1553187523351_0006_01_000005",
"ip": "172.26.81.14",
"launch_time": 1553266732359,
"state": "READY"
},
{
"bare_host": "pandrade-4.openstacklocal.com",
"component_instance_name": "regionserver-0",
"hostname": "regionserver-0.hbase.ambari-qa.OPENSTACKLOCAL.COM",
"id": "container_e21_1553187523351_0006_01_000003",
"ip": "172.26.81.15",
"launch_time": 1553266732358,
"state": "READY"
},
{
"bare_host": "pandrade-3.openstacklocal.com",
"component_instance_name": "regionserver-1",
"hostname": "regionserver-1.hbase.ambari-qa.OPENSTACKLOCAL.COM",
"id": "container_e21_1553187523351_0006_01_000004",
"ip": "172.26.81.13",
"launch_time": 1553266732358,
"state": "READY"
}
],
"dependencies": [],
"launch_command": "sleep 15; /usr/hdp/current/hbase-regionserver/bin/hbase regionserver start",
"name": "regionserver",
"number_of_containers": 3,
"quicklinks": [],
"resource": {
"additional": {},
"cpus": 1,
"memory": "512"
},
"restart_policy": "ALWAYS",
"run_privileged_container": false,
"state": "STABLE"
},
{
"artifact": {
"id": "pandrade-4:5000/hbase_local",
"type": "DOCKER"
},
"configuration": {
"env": {
"HADOOP_HOME": "/usr/hdp/3.1.0.0-78/hadoop",
"HBASE_LOG_DIR": "var/log/hbase"
},
"files": [
{
"dest_file": "/etc/hbase/conf/hbase-site.xml",
"properties": {
"hbase.cluster.distributed": "true",
"hbase.master.hostname": "hbasemaster-0.${SERVICE_NAME}.${USER}.${DOMAIN}",
"hbase.master.info.port": "16010",
"hbase.rootdir": "${SERVICE_HDFS_DIR}/hbase",
"hbase.zookeeper.quorum": "${CLUSTER_ZK_QUORUM}",
"zookeeper.znode.parent": "${SERVICE_ZK_PATH}"
},
"type": "XML"
},
{
"dest_file": "/etc/hadoop/conf/core-site.xml",
"properties": {},
"src_file": "core-site.xml",
"type": "TEMPLATE"
},
{
"dest_file": "/etc/hadoop/conf/hdfs-site.xml",
"properties": {},
"src_file": "hdfs-site.xml",
"type": "TEMPLATE"
}
],
"properties": {
"docker.network": "host"
}
},
"containers": [
{
"bare_host": "pandrade-4.openstacklocal.com",
"component_instance_name": "hbaseclient-0",
"hostname": "hbaseclient-0.hbase.ambari-qa.OPENSTACKLOCAL.COM",
"id": "container_e21_1553187523351_0006_01_000006",
"ip": "172.26.81.15",
"launch_time": 1553266732370,
"state": "READY"
}
],
"dependencies": [],
"launch_command": "sleep infinity",
"name": "hbaseclient",
"number_of_containers": 1,
"quicklinks": [],
"resource": {
"additional": {},
"cpus": 1,
"memory": "512"
},
"restart_policy": "ALWAYS",
"run_privileged_container": false,
"state": "STABLE"
}
],
"configuration": {
"env": {
"HADOOP_HOME": "/usr/hdp/3.1.0.0-78/hadoop",
"HBASE_LOG_DIR": "var/log/hbase"
},
"files": [
{
"dest_file": "/etc/hadoop/conf/core-site.xml",
"properties": {},
"src_file": "core-site.xml",
"type": "TEMPLATE"
},
{
"dest_file": "/etc/hadoop/conf/hdfs-site.xml",
"properties": {},
"src_file": "hdfs-site.xml",
"type": "TEMPLATE"
},
{
"dest_file": "/etc/hbase/conf/hbase-site.xml",
"properties": {
"hbase.cluster.distributed": "true",
"hbase.master.hostname": "hbasemaster-0.${SERVICE_NAME}.${USER}.${DOMAIN}",
"hbase.master.info.port": "16010",
"hbase.rootdir": "${SERVICE_HDFS_DIR}/hbase",
"hbase.zookeeper.quorum": "${CLUSTER_ZK_QUORUM}",
"zookeeper.znode.parent": "${SERVICE_ZK_PATH}"
},
"type": "XML"
}
],
"properties": {
"docker.network": "host"
}
},
"id": "application_1553187523351_0006",
"kerberos_principal": {},
"lifetime": 10200,
"name": "hbase",
"quicklinks": {
"HBase Master Status UI": "http://hbasemaster-0.hbase.ambari-qa.OPENSTACKLOCAL.COM:16010/master-status"
},
"state": "STABLE",
"version": "2.0.2.3.1.0.0"
}Our Hbase service is stable and all docker containers in Ready state
Hbase master UI:
References:
https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/Configurations.html
http://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/yarn-service/RegistryDNS.html
https://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html
Files are also available in the following GitHub repo:
Announcements
What's New @ Cloudera
What's New @ Cloudera
What's New @ Cloudera
What's New @ Cloudera
What's New @ Cloudera
