About dankenstein

dankenstein · ‎09-28-2023

We are using CDSW version 1.10.3.38240717 (xac2a42d). We cannot use the jupyterlab images when creating a job but we need them. Only Workbench images are available. We have spark magic notebooks that need to be put on a scheduler. Any advise would be appreciated.

dankenstein · ‎05-25-2023

We are running CDSW and CDP. Our runtime docker images have tools integrated like beeline, spark, hdfs, ect and they all point to HDP, built into the image. We integrated these commands into our data processing and need the images to be updated to point to our new CDP install. Any advice is appreciated and we are having doubts that Cloudera will be motivated to provide a quick response as they do not seem to really care about CDSW as a product. We are thinking it will be up to us to correct the configuration.

dankenstein · ‎09-15-2022

We have been utilizing CDSW jobs for the past couple years and have users moving onto other projects. The only way that we can find to run the jobs under someone else's account is to delete the jobs and recreate them with the new user. It is insanely annoying and keeps resulting in imperfect recreations. Is there a way to do this without deleting/recreating?

dankenstein · ‎03-16-2021

What our organization did is we have base project that we created that we get everyone the pull from that has everything set up. It is really not ideal but it was the best we could come up with given our control over the product. You are correct that the .jupyter dir does not exist until jupyter-notebooks is run and we could not stick it in any directory to get copied on the docker build. We actually had upgraded over the weekend for other reasons and i tested out the auto kill for jupyter. If you are a CLOUDERA rep or a rep reads this please understand that the main reason our organization chose CDSW is because of this functionality and the ability to edit the docker build. We have been through Anaconda Enterprise, IBM Watson studio, and even tried to run JupyterHub. Our organization has 150+ data scientists/ analysts and they are all irresponsible when it comes to stopping their sessions. Anaconda was by far the worst performing product/company to work with for support. Now with CDSW we can propagate project configuration/ tutorial scripts/ spark and hive config from the top down to all projects via the docker build and we love it. Below are the contents of jupyter-notebook-config.py located in .juypter, it worked for us. , c.NotebookApp.shutdown_no_activity_timeout = 3600 c.MappingKernelManager.cull_idle_timeout = 2600

dankenstein · ‎03-04-2021

CDSW version 1.7 I need to add a file to every image that cdsw builds. jupter_notebook_config.py will need to be placed in the in the .jupyter directory of every project. This is because our organization needs the cdsw sessions to be culled after inactivity and jupyter notebooks prevents the built in idle timeout from functioning.

Online	Offline
Last Visited	‎11-01-2023 10:00 AM

Member Since	‎03-04-2021 07:07 AM
Last Visited	‎11-01-2023 10:00 AM
Posts	5

Cloudera Community

Cloudera Community

Re: Add file to base CDSW image build.

Cannot run jobs with CDSW JupyterLab docker image.

New CDSW runtime images for CDP.

Is there any way to change the user for a CDSW job...

Re: Add file to base CDSW image build.

Add file to base CDSW image build.