Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

In CDSW, how do I import a Scala function/class from a separate file in my project?

avatar
New Contributor

In Data Science Workbench, I would like to create one file with a number of types/functions defined. Then, in a separate file, I'd like to be able to import those types/functions (the first file is essentially acting as a library that can be shared among many other scripts.) How do I go about this? In the docs, I see a mention of it being possible, but no example and the few obvious things I've tried haven't worked, so I'm clearly overlooking something simple.

 

Thanks in advance for any help!

 

- J.

1 ACCEPTED SOLUTION

avatar
New Contributor

Thank you, Paul. That's what I've been doing up to this point. I was hoping (but not expecting) there might be a way to import definitions from file A without going through the "Run All" step. The reason is to be able to kick off an experiment that runs file B without having to manually copy definitions from A to B.

 

I'm guessing that's not possible, which is fine. I just didn't want to start copying stuff from file to file if it could be avoided. For more established code, I'll probably wind up building a jar. I was just hoping for something with a little quicker turnaround time for now.

 

Thanks,

 

- J.

View solution in original post

3 REPLIES 3

avatar
Contributor

Within a CDSW project, you have the option to manually upload files from your computer or import them from a github repository by pulling from it.

 

Once you have created a project, you can access project files from the Overview Page, the Files page, or from within a terminal window if you launch a CDSW session. The project files are automatically loaded to the /home/cdsw directory but you can create additional directories at will.  

 

This quickstart guide will give you some more details on the above steps. https://docs.cloudera.com/documentation/data-science-workbench/1-7-x/topics/cdsw_quickstart.html

 

Once you have created a project, create two scala files by clicking on "new session" and selecting the Scala Engine Kernel. I recommend leaving the default Workbench editor for the purposes of this example. Next,  you can put your methods etc. in your first file, and import them from the second file. Finally, run the second file within the Scala session by clicking "Run" -> "Run All". 

 

I would recommend following this short tutorial example and creating project directories and files in CDSW https://www.cs.helsinki.fi/u/wikla/OTS/Sisalto/examples/html/ch13.html#sec2

avatar
New Contributor

Thank you, Paul. That's what I've been doing up to this point. I was hoping (but not expecting) there might be a way to import definitions from file A without going through the "Run All" step. The reason is to be able to kick off an experiment that runs file B without having to manually copy definitions from A to B.

 

I'm guessing that's not possible, which is fine. I just didn't want to start copying stuff from file to file if it could be avoided. For more established code, I'll probably wind up building a jar. I was just hoping for something with a little quicker turnaround time for now.

 

Thanks,

 

- J.

avatar
New Contributor

I think this answer may have been edited since I originally looked at it. I don't recall seeing that link to the Scala tutorial and the suggestion that importing should work.

 

This still does not work for me. If I follow the code in that tutorial, I get:

 

import bobsdelights.Fruit
Name: Unknown Error
Message: <console>:80: error: not found: value bobsdelights
       import bobsdelights.Fruit
              ^

 

Have you confirmed that you are able to import local Scala code from one file to another without first executing that first file in the shell? The intention is to be able to run an experiment and I can only submit one script via the experiment UI.