question Re: [Newbie] Is it possible to write/run Spark code in JAVA on HDP2.4 (Centos 6.7) in Archives of Support Questions (Read Only)

[Newbie] Is it possible to write/run Spark code in JAVA on HDP2.4 (Centos 6.7)

navpap_1029 — Mon, 18 Apr 2016 09:03:57 GMT

I am a newbie to ambari and HDP, currently playing on HDP2.4, I have gone through a couple of sample exercises mentioned in hortonworks pages, given my interest/focus is on Spark, I understand that I can write spark code in Zepplin-notebook, the exercises that I went through were already in Scala.

Given that I hail from java background,

1) Is it possible to write/run spark code in java on zepplin or any other workaround?

2) Could anyone link me to good starter-intermediate spark examples/questions (not necessarily Java)?

Many thanks in advance,

Naveen

Re: [Newbie] Is it possible to write/run Spark code in JAVA on HDP2.4 (Centos 6.7)

pminovic — Mon, 18 Apr 2016 12:56:56 GMT

For Spark on HDP, please check HDP Spark Guide. For general Spark programming check Spark Programming Guide with all examples given in Scala, Java, and Python. If you want to program in Java you need a development environment like Eclipse or IntelliJ.

Re: [Newbie] Is it possible to write/run Spark code in JAVA on HDP2.4 (Centos 6.7)

phargis — Tue, 19 Apr 2016 00:06:09 GMT

Zeppelin comes with a long list of interpreters (including Spark/Scala, Python/Pyspark, Hive, Cassandra, SparkSQL, Phoenix, Markdown and Shell), which basically provide language binding to run code that you type into a notebook cell. Currently, the list of interpreters does not include Java, so you will need to compile your code first and build a jar file, which can be submitted to Spark via spark-submit, as described here:

http://spark.apache.org/docs/latest/submitting-applications.html

Re: [Newbie] Is it possible to write/run Spark code in JAVA on HDP2.4 (Centos 6.7)

navpap_1029 — Sat, 23 Apr 2016 08:04:59 GMT

Thanks for the reply, gives me a good start, on that node, is it possible to add GUI to the HDP VM and install intelliJ there?

Re: [Newbie] Is it possible to write/run Spark code in JAVA on HDP2.4 (Centos 6.7)

navpap_1029 — Sat, 23 Apr 2016 08:09:08 GMT

good to know, thanks for the sharing that information.