Reply
New Contributor
Posts: 1
Registered: ‎01-29-2018

Data Science Workbench installation instructions 1.3.X - clarification needed

[ Edited ]

Hi, I'm very new to Cloudera. I need to setup CDSW by using CSD. But I'm not quite sure how many node will I need in oder to set up test environment. I have to setup and build the environment on my own from the start. Can anyone please suggest?

 

- How many node will I need? As I understand 3 (1 for cm, 1 cdsw master, 1  cdsw worker)

- CM node will only store roles, and cdsw will store only its own packages (including agent)?

- Can cdsw master and worker run on the same node? 

- And from this guide that I have to config dns wildcard subdomain, do I need to create subdomain first?

- Anything I miss or I should know?

 

I was trying to setup once on CDSW 1.2.X with two node 1 as master with cm, the other one was just worker. The result was I cannot init or start the cdsw web. Now I will try again from the begining, please help!

Highlighted
Cloudera Employee
Posts: 327
Registered: ‎03-23-2015

Re: Data Science Workbench installation instructions 1.3.X - clarification needed

- How many node will I need? As I understand 3 (1 for cm, 1 cdsw master, 1 cdsw worker)

Typically you will need a CDH cluster and CDSW cluster. For CDH cluster, ideally you need at least 2-3 nodes, but one big host can also run fine for CDH. For CDSW cluster, one node is enough for testing as I have done it before (master and worker on the same node), but you probably will need a bigger host for docker to run smoothly.

- CM node will only store roles, and cdsw will store only its own packages (including agent)?

From CDSW 1.3, you can have CDSW managed by CM, so CDSW host will also have CM agents running, on top of its own parcels. But if you install using packages, then it will run outside of CM. For the CM/CDH host, CM packages and CDH parcels will be installed.

- Can cdsw master and worker run on the same node?
Yes

- And from this guide that I have to config dns wildcard subdomain, do I need to create subdomain first?
I believe so.

- Anything I miss or I should know?
The Required Pre-Installation Steps are outlined here already:
https://www.cloudera.com/documentation/data-science-workbench/latest/topics/cdsw_install.html#pre_in...

which is in the link you provided.
Announcements