Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎06-14-2019

How to use local IDE and run Python code in Cloudera?

Hi,

 

I am new to using Cloudera. I wonder if there is a way to write Python code in e.g. Atom or Spyder and execute that code on the Cloudera engine?

 

Basically I wonder if I can run Python with comparable performance locally as in Cloudera Workbench, with such a method.

 

Best regards!

Cloudera Employee
Posts: 76
Registered: ‎11-16-2015

Re: How to use local IDE and run Python code in Cloudera?

[ Edited ]

Hello @Data_Dog 

 

Welcome! 

What you are trying to achieve is not there yet in the existing versions (latest is 1.5 as of writing). But, the good news is it will be there in the upcoming CDSW 1.6 version which provides support for local editors (eg PyCharm which supports SSH) allowing remote execution on CDSW and also file sync from local editors to Cloudera DataScience Workbench over SSH. CDSW 1.6 also provides lot of other enhancements including support for 3rd party editors. If you'd like to know more about the upcoming release please see https://www.cloudera.com/about/events/webinars/virtual-event-ml-services-cdsw.html 

 

Thank you,

Amit

New Contributor
Posts: 4
Registered: ‎06-28-2019

Re: How to use local IDE and run Python code in Cloudera?

Get 404 on link to upcoming release.
New Contributor
Posts: 4
Registered: ‎06-28-2019

Re: How to use local IDE and run Python code in Cloudera?

Any plans for supporting "real" IDEs such as Eclipse or Netbeans? 

Cloudera Employee
Posts: 1
Registered: ‎04-28-2017

Re: How to use local IDE and run Python code in Cloudera?

Support won't be provided explicitly by individual editor or IDE. Instead the remote ssh functionality in CDSW 1.6 is provided for you to use existing remote connection approaches. I've not done discovery on these IDEs myself, but check out:

https://help.eclipse.org/kepler/index.jsp?topic=%2Forg.eclipse.rse.doc.user%2Fgettingstarted%2Fg_sta...

 

... for netbeans, I couldn't find an obvious link.