Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Java MapReduce client connecting to Hortonworks VM

avatar
New Contributor

I would like to be able to create a simple MapReduce application and then run it locally on my desktop by connecting to the Hortonworks VM (also running on my desktop). Are there any examples or guidelines of what configuration options I need to change in mapred-site.xml, core-site.xml, yarn-site.xml, etc? I have tried a few things from stackoverflow, including setting mapreduce.framework.name as referenced below, but I am seeing the following error when I start the job:

DEBUG [main] (UserGroupInformation.java:1661) - PrivilegedActionException as:bc (auth:SIMPLE) cause:java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. Exception in thread "main" java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.

I realize I could just scp it over, compile, and submit the job directly on the VM but it would be nice to iterate locally in Eclipse and avoid the churn of copy/compile/jar/submit job every edit.

1 ACCEPTED SOLUTION

avatar
Master Mentor

you will need the client jars and all the xml files on your local machine. In all honesty, I think it's easier to scp stuff over and run it on Sandbox @Barrett Cervenka. You can also try this tutorial by @Ali Bajwa.

View solution in original post

3 REPLIES 3

avatar
Master Mentor

you will need the client jars and all the xml files on your local machine. In all honesty, I think it's easier to scp stuff over and run it on Sandbox @Barrett Cervenka. You can also try this tutorial by @Ali Bajwa.

avatar
New Contributor

Yea, I think I have come to the same conclusion that it would just be easier to copy over to the sandbox. Thanks for the tutorial link though. I had not come across that yet.

avatar
Master Mentor