Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

yarn java configuration issue

yarn java configuration issue

New Contributor

I've built a new cluster and all seems right but yarn/mapreduce jobs won't run. Doing a simple wordcount I get...

 

17/09/05 22:44:21 INFO client.RMProxy: Connecting to ResourceManager at cldradn101.dassvm.nccs.nasa.gov/10.71.11.21:8032
17/09/05 22:44:22 INFO input.FileInputFormat: Total input paths to process : 1
17/09/05 22:44:22 INFO mapreduce.JobSubmitter: number of splits:8
17/09/05 22:44:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1504648306083_0003
17/09/05 22:44:22 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/yarn/.staging/job_1504648306083_0003
java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext.setAMContainerResourceRequests(Ljava/util/List;)V
at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:579)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:315)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:244)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)

 

 

Can someone suggest where to look for I assume is a java issue? Thanks in advance!

2 REPLIES 2

Re: yarn java configuration issue

Master Guru
What CDH version is your installed cluster, and what version are you compiling/bundling with your application jar?

The method specifically that its trying to call on the client-side is a new one in CDH 5.12.0. However, given that its not finding that method yet referencing it, it appears that you may be bundling an older or newer (depending on what version your cluster runs) CDH/Hadoop dependencies with your application jar which is not a good idea (the platform provides these by default on its classpath).

Does an inbuilt example work normally? Try with:

hadoop org.apache.hadoop.examples.WordCount <in-dir> <out-dir>

P.s. Ensure you use and reference CDH dependencies in your project for Apache Hadoop (hadoop-client), not the upstream ones. The guide to do this is at https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh5_maven_repo.html

Re: yarn java configuration issue

Explorer

Hi Harsh,

 

we are using CDH 5.12.2 and facing the same issue while executing the sqoop import job and also tried the same  example as you mentioned below. Below example also didn't work and got the same error. 

 

hadoop org.apache.hadoop.examples.WordCount <in-dir> <out-dir> 

 

 

Is there any workaround for this issue? Please let me know. 

 

Regards,

Sandeep. v

Don't have an account?
Coming from Hortonworks? Activate your account here