issue creating disk Docker image Block device, DNS wildcard, Spark2 for CDSW installation

[#1]

Hello,

Could someone help me in creating a Docker Block Device on my CDSW Master node. I am getting the below errors:

Checking if [conntrack-tools.x86_64] is installed...OK
Setting up docker storage.

ERROR:: Error in pvcreate for [/dev/sdb]: 5

ERROR:: Unable to setup docker storage.: 5

ERROR:: Unable to create storage for docker.: 5

Installed:
  conntrack-tools.x86_64 0:1.4.4-7.el7                                          

Dependency Installed:
  libnetfilter_cthelper.x86_64 0:1.0.0-11.el7                                   
  libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7                                   
  libnetfilter_queue.x86_64 0:1.0.2-2.el7_2                                     

Complete!
Setting up docker storage.

ERROR:: Entries in DOCKER_BLOCK_DEVICES must only be block devices: [/dev/sde]: 1

ERROR:: Unable to create storage for docker.: 1

here's my server disks look like . shoud this be done only on Master CDSW node or even in the worker CDSW node ?

I have a 2TB sdb disk which can I give for a new /dev/sdb3 something ... how do I do this ? can you please share the commands for creating a  new disk of size 500GB-1TB for Docker Image block device without mounting it. 

Also, would that docker image space be used later or I can follow this method of creating a files as a docker image block given in this solution. not sure if it would work and recommended. 

https://community.cloudera.com/t5/Support-Questions/CDSW-installation-Entries-in-DOCKER-BLOCK-DEVICE...

Also, please let me know if setting up DNS wildcard is adding the *domain name on /etc/hosts or any other DNS file of the linux server. and is it only on the Master CDSW and need to do the same on the worker CDSW node as well. 

Also, should Spark2 be installed manually or could be added as a service from the cluster but dont see the option in the cluster to add as a service. where are the parcels to downloaded for spark2 .. or should Install it like its shown in this doc

https://docs.cloudera.com/cdsw/1.9.0/installation/topics/cdsw-configure-apache-spark-2-cdh6-or-7.htm...

Please help , I'm stuck with this.. Thanks! 

[#2]

The CDSW installer will take care of the format and mount of the Docker Block Device as indicated here: https://docs.cloudera.com/cdsw/1.9.1/installation/topics/cdsw-docker-block-device.html

You need to setup wildcard DNS subdomain with your DNS server

https://docs.cloudera.com/cdsw/1.9.1/installation/topics/cdsw-set-up-a-wildcard-dns-subdomain.html

For Spark2, it should be installed in the CDH/CDP hosts, install Gateway service in CDSW node(s), details: 

https://docs.cloudera.com/cdsw/1.9.1/installation/topics/cdsw-configure-apache-spark-2-cdh6-or-7.htm...

[#3]

It looks like you have CDSW configured to use /dev/sdb for your docker block device:

ERROR:: Error in pvcreate for [/dev/sdb]: 5

 But somehow you also have /dev/sde set:

ERROR:: Entries in DOCKER_BLOCK_DEVICES must only be block devices: [/dev/sde]: 1

Check your configuration for DOCKER_BLOCK_DEVICE in CM, it is possible that you have multiple disks set for some reason. (People often do this when they have multiple nodes and the disks are different on each node. You need to use the role group feature to do this instead of listing all of the disks.)

A docker block device will be created on the master AND worker, so you should make sure that both machines hav a /dev/sde (or /dev/sdb) that is free and formatted. OR use role groups to set master to be dev/sdb and worker to be /dev/sde.  

>Also, please let me know if setting up DNS wildcard is adding the *domain name on /etc/hosts or any other DNS file of the linux server

No. You need to go into the DNS server itself and add an entry for a wildcard DNS for your domain. It should be *.[your-cdsw-url]. You cannot use /etc/host entries; CDSW ignores /etc/host in containers. I have never set up a DNS server so I can't really help with this.