07-24-2017 01:08 AM
I've successfully installed CDH 5.11.1 with Cloudera Manager on a 4 node cluster. Here is my setup:
node1.cirro.com | 8GB RAM | 70GB HDD | 192.168.1.101 | master node2.cirro.com | 4GB RAM | 70GB HDD | 192.168.1.102 | slave node3.cirro.com | 4GB RAM | 70GB HDD | 192.168.1.103 | slave node4.cirro.com | 4GB RAM | 70GB HDD | 192.168.1.104 | slave
After creating a cluster with Spark installed in it, I've immediately downloaded cloudera data science workbench 1.0.1 rpm and installed it in my master node.
After installation, I edited /etc/cdsw/config/cdsw.conf and replaced the following:
DOMAIN="cdsw.cirro.com" MASTER_IP="192.168.1.101" DOCKER_BLOCK_DEVICES=""
I don't know what to change in the DOCKER_BLOCK_DEVICES part, so I executed
and error came out:
DOCKER_BLOCK_DEVICES is not set
which is to be expected, however, I don't know what to include. I suspect I should spin up a new HDD with at least 500 GB and set it as the block device?
Plus, I'm not sure if the direction of the setup (tho incomplete) is correct, is it on the right direction?
disclaimer: I'm a newbie with Cloudera and it took me almost a month to install CDH and CM. Bear with me good sirs! :) Thanks!
07-24-2017 08:43 AM
As I understand you tried to install the product on your CDH master host which is not supported.
Cloudera Data Science Workbench hosts need to be added to your CDH cluster as gateway hosts as an extension of your CDH cluster. You can find the recommended HW configuration here:
After you got the extra host you can add the extra block devices there. This should give you context what do we expect there: