Member since
07-09-2015
68
Posts
24
Kudos Received
12
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
8996 | 11-23-2018 03:38 AM | |
2310 | 10-07-2018 11:44 PM | |
2949 | 09-24-2018 12:09 AM | |
4669 | 09-13-2018 02:27 AM | |
2992 | 09-12-2018 02:27 AM |
07-13-2017
05:19 AM
Hi Chris, You can use the SparkSession object to query your Hive tables. You don't need to create a HiveContext. spark.catalog.listTables.show()
spark.sql("describe database default").show
spark.sql("describe formatted default.mytable").show I don't think that this change would fix your issue. This doesn't seem like a problem with your CDSW installation. Could you execute the same commands from a Session terminal or directly on the CDSW host machine using the spark2-shell cli tool to verify this? Could you double check that a Hive gateway is deployed to your CDSW host? Can you show that the table exists from eg. beeline? Regards, Peter
... View more
07-13-2017
04:06 AM
Hello Chris, What are you trying to achieve? Why are you trying to run in local mode? Can you use the expected yarn client mode instead? From the documentation: "Unlike PySpark or Sparklyr, you can access a SparkContext assigned to the spark (SparkSession) and sc(SparkContext) objects on console startup, just as when using the Spark shell." https://www.cloudera.com/documentation/data-science-workbench/latest/topics/cdsw_spark_and_scala.html Can you access to your hive tables using the spark SparkSession object? Regards, Peter
... View more
07-06-2017
07:59 AM
Hello, This is weird. Could you try a manual docker pull? docker.repository.cloudera.com/cdsw/1.0.1/web:052787a Thanks, Peter
... View more
07-05-2017
01:32 AM
Hi NES, I'm glad that your issue is resolved! Could you let us know what was the issue for the other topic followers? Thanks, Peter
... View more
07-05-2017
12:45 AM
Hi NES, I see that the web pods are stuck in ContainerCreating state. I would try to submit a kubectl describe command on one of the web pods and check if we have any associated events. $ kubectl describe pod web-3826671331-5b7wk If there are no events I would check the kubelet logs, maybe it shows the reason why are we stuck: # get the kubelet logs in less $ journalctl -u kubelet # tail the kubelet logs $ journalctl -u kubelet -o cat -l -f Regards, Peter
... View more
07-04-2017
05:48 AM
Hi NES, Could you share the full 'cdsw status' output? I would like to see what system pods are not ready. There is a db-migrate-XXXXX-xxxxx Kubernetes job which should make sure that the tables are created. When you submit the 'cdsw status' command, can you see this job in the output? Did you see a Completed status for this? When you changed the cdsw configuration, did you submit a 'cdsw reset' before retrying the 'cdsw init'? If not, you will need to do a 'cdsw reset' and 'cdsw init' again. 'cdsw logs' generates a tar.gz file with different diagnostic information. If you want to validate your cluster, I recommend to use the 'cdsw validate' instead. Thanks, Peter
... View more
06-22-2017
02:25 AM
Hi Krishna, There is the true flag under the Stateful column, just it's not rendered nicely, please disregard the previous post. Could you try to restart your cdsw application and upload the `cdsw status` command output again? We shouldn't see multiple database pods there. Thanks, Peter
... View more
06-22-2017
02:20 AM
Hi Krishna, One issue we see is the master node is missing the stateful true flag. Something might went wrong with your installation. The code which would put the stateful tag on the node is using the `hostname` command output. Could you check if the `hostname` output matches the node name you see in the `cdsw status` output? I would start by doing `cdsw stop` and `cdsw start` on the master node and see if the stateful flag gets there. If this is not working you will probably need to do a `cdsw reset` and `cdsw init` for the master and workers also. Regards, Peter
... View more
06-21-2017
04:47 AM
Hi Krishna, We have a known issue with running the 'cdsw status' command on a worker host. We are planning to fix this for the next release. In the meantime, please submit the 'cdsw status' command on your master host to see the status of your workbench. Could you share your status command output from the master host with us, as it might give us some context what might be the issue? Are you sure you followed the install steps consistently on the worker hosts, e.g. submitted the 'cdsw enable <IPv4_address_of_worker>' on all worker? Thanks, Scotty
... View more
06-16-2017
03:20 AM
Hi Krishna, It seems this is the second time you are trying to submit the cdsw join command. The join should complete fully, in the case of errors, you will need to reset which will clean up any invalid state. Please try the recommendation: $ cdsw reset $ cdsw join I'm adding the documentation steps for reference: https://www.cloudera.com/documentation/data-science-workbench/latest/topics/cdsw_install.html#install_on_workers Regards, Peter
... View more
- « Previous
- Next »