Eclipse to Sandbox

[#1]

Hi,

I am new to HDP and hadoop.I managed to install HDP 2.3 sandbox on Virtual box.

I tried a few sample programs and they are working fine from the sandbox.

I have installed Eclipse with Scala in my Windows machine.

At present ,I use SBT and package my application and deploy the jar in the HDP Sandbox for execution.

I would like to execute programs from my Eclipse against the HDP sandbox directly instead of packaging it each and every time.

A sample code which I am trying to modify

val conf = new SparkConf().setAppName(“Simple Application”).setMaster(“local[2]”).set(“spark.executor.memory”,”1g”)

I guess , I have to change the local[2] to the master node / yarn cluster url.

How do I get the url from the sandbox ?

Any other configurations which has to be done on the Virtual box or on my code ?

[#2 · Accepted solution]

You have to switch master to "yarn-client" . I believe you will also need to have HADOOP_CONF_DIR and YARN_CONF_DIR and probably make a copy of those configs from sandbox to a directory that is on your classpath in eclipse. Sorry I have not done it personally

[#3]

Hi @Saptak Sen,

I want the same way... so execute programs from Eclipse under my HDP Sandbox 2.5.

Please let me know step by step all configurations to be applied from my windows environment

Thanks