Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Java MapReduce client connecting to Hortonworks VM

New Contributor

I would like to be able to create a simple MapReduce application and then run it locally on my desktop by connecting to the Hortonworks VM (also running on my desktop). Are there any examples or guidelines of what configuration options I need to change in mapred-site.xml, core-site.xml, yarn-site.xml, etc? I have tried a few things from stackoverflow, including setting mapreduce.framework.name as referenced below, but I am seeing the following error when I start the job:

DEBUG [main] (UserGroupInformation.java:1661) - PrivilegedActionException as:bc (auth:SIMPLE) cause:java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. Exception in thread "main" java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.

I realize I could just scp it over, compile, and submit the job directly on the VM but it would be nice to iterate locally in Eclipse and avoid the churn of copy/compile/jar/submit job every edit.

1 ACCEPTED SOLUTION

Mentor

you will need the client jars and all the xml files on your local machine. In all honesty, I think it's easier to scp stuff over and run it on Sandbox @Barrett Cervenka. You can also try this tutorial by @Ali Bajwa.

View solution in original post

3 REPLIES 3

Mentor

you will need the client jars and all the xml files on your local machine. In all honesty, I think it's easier to scp stuff over and run it on Sandbox @Barrett Cervenka. You can also try this tutorial by @Ali Bajwa.

New Contributor

Yea, I think I have come to the same conclusion that it would just be easier to copy over to the sandbox. Thanks for the tutorial link though. I had not come across that yet.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.