In CDSW, how do I import a Scala function/class from a separate file in my project?

Question

In Data Science Workbench, I would like to create one file with a number of types/functions defined. Then, in a separate file, I'd like to be able to import those types/functions (the first file is essentially acting as a library that can be shared among many other scripts.) How do I go about this? In the docs, I see a mention of it being possible, but no example and the few obvious things I've tried haven't worked, so I'm clearly overlooking something simple.

Thanks in advance for any help!

- J.

jmcconnell · Accepted Answer

Thank you, Paul. That's what I've been doing up to this point. I was hoping (but not expecting) there might be a way to import definitions from file A without going through the "Run All" step. The reason is to be able to kick off an experiment that runs file B without having to manually copy definitions from A to B.

I'm guessing that's not possible, which is fine. I just didn't want to start copying stuff from file to file if it could be avoided. For more established code, I'll probably wind up building a jar. I was just hoping for something with a little quicker turnaround time for now.

Thanks,

- J.

pauldefusco · Answer

Within a CDSW project, you have the option to manually upload files from your computer or import them from a github repository by pulling from it.

Once you have created a project, you can access project files from the Overview Page, the Files page, or from within a terminal window if you launch a CDSW session. The project files are automatically loaded to the /home/cdsw directory but you can create additional directories at will.

This quickstart guide will give you some more details on the above steps. https://docs.cloudera.com/documentation/data-science-workbench/1-7-x/topics/cdsw_quickstart.html

Once you have created a project, create two scala files by clicking on "new session" and selecting the Scala Engine Kernel. I recommend leaving the default Workbench editor for the purposes of this example. Next, you can put your methods etc. in your first file, and import them from the second file. Finally, run the second file within the Scala session by clicking "Run" -> "Run All".

I would recommend following this short tutorial example and creating project directories and files in CDSW https://www.cs.helsinki.fi/u/wikla/OTS/Sisalto/examples/html/ch13.html#sec2

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.