Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Local configuration of pyCharm - Spark

Highlighted

Local configuration of pyCharm - Spark

New Contributor

Hi,

I have a question regarding the setup of a development environment.

I’m using an AWS where I installed the last version of Hortonworks and wants to do some POC around kafka and spark streaming.

As the access to AWS is not free I wanted to create a dev env on my local laptop based on PyCharm and spark 1.6.

Usually I use a maven repository but I’m looking for a simpler solution. Can this works:

  1. Install localy the exact spark version than AWS
  2. Setup the PyCharm to point to the local spark
  3. Do the development locally then deploy and run the code on the AWS

Thanks

Don't have an account?
Coming from Hortonworks? Activate your account here