Reply
New Contributor
Posts: 3
Registered: ‎05-19-2017

cloudera data science work bench

Hai team,

 

we have installed clodera data science  work bench on centos7/spark 1.6 .everything went good but still there is an error shows in the logs 

 

Unexpected line format: [{"code":"UNUSED_DEFINITION","message":"Definition is not used: #/definitions/User","path":["definitions","User"]},{"code":"UNUSED_DEFINITION","message":"Definition is not used: #/definitions/Organization","path":["definitions","Organization"]}] 2

Unexpected line format: connect.multipart() will be removed in connect 3.0

Unexpected line format: visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives

Unexpected line format: connect.limit() will be removed in connect 3.0

Unexpected line format: connect.multipart() will be removed in connect 3.0

Unexpected line format: visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives

Unexpected line format: connect.limit() will be removed in connect 3.0

Unexpected line format: connect.multipart() will be removed in connect 3.0

Unexpected line format: visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives

Unexpected line format: connect.limit() will be removed in connect 3.0

Unexpected line format: connect.multipart() will be removed in connect 3.0

Unexpected line format: visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives

Unexpected line format: connect.limit() will be removed in connect 3.0

Producing redacted logs tarball...

 

 

 

 

When I found status and validate ,it shows everything is good.

 

[root@or1010051029029 ~]# cdsw validate

Checking services...

Checking if docker is active and enabled

Checking if docker is responsive

Checking if kubelet is active and enabled

Check if docker monitor is active

Testing networking setup...

Check if kubelet iptables rules exist

Check that firewalld is disabled

Check configuration file...

Checking master node filesystem configuration...

Checking kubernetes

Checking system pods

Checking application pods exist

Checking application pods are running

Checking web pods have access to the databases

Checking connectivity to application services

Checking connectivity over ingress

 

No errors detected.

 

If you experience issues but the validation passes, please capture logs from

all nodes using "cdsw logs".

 

[root@or1010051029029 ~]# cdsw status

Cloudera Data Science Workbench Status

 

Service Status

docker: active

kubelet: active

nfs: active

Checking kernel parameters...

 

Node Status

NAME                             STATUS    AGE       STATEFUL

or1010051029029.corp.adobe.com   Ready     9m        true

 

System Pod status

NAME                                                     READY     STATUS    RESTARTS   AGE

dummy-2088944543-gkn7q                                   1/1       Running   0          9m

etcd-or1010051029029.corp.adobe.com                      1/1       Running   0          9m

kube-apiserver-or1010051029029.corp.adobe.com            1/1       Running   0          9m

kube-controller-manager-or1010051029029.corp.adobe.com   1/1       Running   0          9m

kube-discovery-1150918428-bl6vb                          1/1       Running   0          9m

kube-dns-3873593988-dkkvp                                3/3       Running   0          8m

kube-proxy-ufab1                                         1/1       Running   0          9m

kube-scheduler-or1010051029029.corp.adobe.com            1/1       Running   0          9m

node-problem-detector-v0.1-cxpq0                         1/1       Running   0          7m

weave-net-jj9az                                          2/2       Running   0          9m

 

Cloudera Data Science Workbench Pod Status

NAME                                  READY     STATUS      RESTARTS   AGE       ROLE

cron-3971587342-s8eej                 1/1       Running     0          7m        cron

db-4066525870-98iwb                   1/1       Running     0          7m        db

db-migrate-abec968-wdf8i              0/1       Completed   0          7m        db-migrate

engine-deps-r7o1d                     1/1       Running     0          7m        engine-deps

ingress-controller-2976678207-sqipw   1/1       Running     0          7m        ingress-controller

livelog-2494298876-emgju              1/1       Running     0          7m        livelog

reconciler-577027981-vv68d            1/1       Running     0          7m        reconciler

spark-port-forwarder-6lsp2            1/1       Running     0          7m        spark-port-forwarder

web-1304125449-cruk9                  1/1       Running     0          7m        web

web-1304125449-o8vc8                  1/1       Running     0          7m        web

web-1304125449-y4450                  1/1       Running     0          7m        web

 

Cloudera Data Science Workbench is ready!

 

can you please evalauate it.

 

 

thanks,

harish.

Cloudera Employee
Posts: 40
Registered: ‎04-28-2017

Re: cloudera data science work bench

From "cdsw status" it appears that the application is up and running correctly.  If you open your browser and point it to the domain you configured as DOMAIN, you should be able to see the application, assuming you have configured your wildcard DNS entries for DOMAIN correctly.

 

The error messages in the logs do not indicate a problem.  Those errors simply indicate that those log lines are not compliant with our redaction tool and will be dropped.

 

I hope that is helpful.  Let us know if you continue to have issues.


Tristan

New Contributor
Posts: 3
Registered: ‎05-19-2017

Re: cloudera data science work bench

 hai trist,

thnaks fro your response.

could you help me in configuring dns wild card entry..? i dont have  correct idea where to configure..?

 

 

 

thanks,

harish

Cloudera Employee
Posts: 40
Registered: ‎04-28-2017

Re: cloudera data science work bench

The documentation for wildcard DNS configuration is here:

 

https://www.cloudera.com/documentation/data-science-workbench/latest/topics/cdsw_install.html#set_up...

 

If you are wondering how to configure a wildcard DNS with your DNS provider, you'll need to consult your DNS provider documentation.  For instance, GoDaddy documentation is here:

 

https://www.godaddy.com/help/setting-up-wildcard-dns-3301

 

I hope that helps.

 

Best,
Tristan

New Contributor
Posts: 3
Registered: ‎05-19-2017

Re: cloudera data science work bench

can you be able to point at which file shouldi edit in linux file system..?or i should i assign to the network team..?


thanks,
harish.
Cloudera Employee
Posts: 40
Registered: ‎04-28-2017

Re: cloudera data science work bench

You'll likely want to talk to a network administrator who manages your organization's DNS records.

 

Best,

Tristan

t5
Explorer
Posts: 16
Registered: ‎07-13-2018

Re: cloudera data science work bench

[ Edited ]

If I understand correctly we need to add both the dns entry (*.cdsw.<domain-name>.com and cdsw.<domain-name>.com) to work on the cdsw as per the documentation. But when I checked with my network team they say they can only add one dns entry(I dont know how far its true??) for the same ip and due to that I am unable to access the cdsw web url as currently the wild card dns entry was added. @tristanzajonc Any advise/suggestion on the same would be great help. Thanks!

Cloudera Employee
Posts: 40
Registered: ‎04-28-2017

Re: cloudera data science work bench

That in general should not be a problem. You can have many DNS names
pointing to the same IP. CDSW does need both, so that it can serve the
root domain as well.

Tristan
t5
Explorer
Posts: 16
Registered: ‎07-13-2018

Re: cloudera data science work bench

@tristanzajonc Thanks for your quick response. Will follow up with my network team to get it done..

Highlighted
t5
Explorer
Posts: 16
Registered: ‎07-13-2018

Re: cloudera data science work bench

[ Edited ]

@tristanzajoncsorry to bother you again.If i am am not wrong cloudera does not provide any sandbox for cdsw along with the CDH cluster. Because I have checked the cdh sandbox(5.13) and it only contains the cdh services not cdsw. If we add the cdsw service then it will create the port conflict and that was reason cloudera asked to install the cdsw in a different node.  Am i correct?

Announcements