IntelliJ / Eclipse Usage Against HDP 2.5 Sandbox

[#1]

This tutorial is great:

https://github.com/hortonworks-gallery/ambari-vnc-service

Eclipse Plugin

https://github.com/winghc/hadoop2x-eclipse-plugin

JDK 7 is best for most use case and Scala 2.10.

Maven and SBT are necessary as well.

Setup your Environment

https://dzone.com/articles/spark-and-scala-resources

https://dzone.com/articles/whats-on-your-laptop

Lots of options:

This is an eclipse project for Hbase Coprocessor

https://github.com/tspannhw/hbasecoprocessor

Artem has a great project for testing

https://github.com/dbist/HBaseUnitTest

Once all the ports are open and not firewalled it’s usually straight forward.

Eclipse to Spark

https://community.hortonworks.com/questions/36354/eclipse-to-sandbox-1.html

https://community.hortonworks.com/questions/32567/scala-with-hive-in-ecplipse-scala.html

Hadoop Eclipse Plugin

https://community.hortonworks.com/questions/10404/hadoop-eclipse-plugin.html

IntelliJ Project for Spark

https://github.com/agilemobiledev/sparkworkshop

https://community.hortonworks.com/questions/31077/how-to-setup-intellij-idea-16-to-run-hortonworks-s...

IntelliJ Settings

https://community.hortonworks.com/questions/37410/recommended-idea-intellij-vmoptions-setting-for-de...

These configuration files must be in project or class path:

• core-site.xml
• hdfs-site.xml
• yarn-site.xml

Add Jars for Access

http://nivemaham.com/index.php/technical/22-java/hadoop/40-how-to-use-ide-for-hadoop-development-wit...

For Apache Kylin development

http://kylin.apache.org/development/dev_env.html

Remote Debugging Spark

https://nicolasmaillard.com/2016/02/06/remote-debugging-201-spark/

Testing with Hadoop MiniClusters

https://github.com/sakserv/hadoop-mini-clusters