yarn java configuration issue

[#1]

I've built a new cluster and all seems right but yarn/mapreduce jobs won't run. Doing a simple wordcount I get...

17/09/05 22:44:21 INFO client.RMProxy: Connecting to ResourceManager at cldradn101.dassvm.nccs.nasa.gov/10.71.11.21:8032
17/09/05 22:44:22 INFO input.FileInputFormat: Total input paths to process : 1
17/09/05 22:44:22 INFO mapreduce.JobSubmitter: number of splits:8
17/09/05 22:44:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1504648306083_0003
17/09/05 22:44:22 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/yarn/.staging/job_1504648306083_0003
java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext.setAMContainerResourceRequests(Ljava/util/List;)V
at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:579)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:315)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:244)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)

Can someone suggest where to look for I assume is a java issue? Thanks in advance!

[#2]

What CDH version is your installed cluster, and what version are you compiling/bundling with your application jar?

The method specifically that its trying to call on the client-side is a new one in CDH 5.12.0. However, given that its not finding that method yet referencing it, it appears that you may be bundling an older or newer (depending on what version your cluster runs) CDH/Hadoop dependencies with your application jar which is not a good idea (the platform provides these by default on its classpath).

Does an inbuilt example work normally? Try with:

hadoop org.apache.hadoop.examples.WordCount <in-dir> <out-dir>

P.s. Ensure you use and reference CDH dependencies in your project for Apache Hadoop (hadoop-client), not the upstream ones. The guide to do this is at https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh5_maven_repo.html

[#3]

Hi Harsh,

we are using CDH 5.12.2 and facing the same issue while executing the sqoop import job and also tried the same  example as you mentioned below. Below example also didn't work and got the same error. 

hadoop org.apache.hadoop.examples.WordCount <in-dir> <out-dir> 

Is there any workaround for this issue? Please let me know. 

Regards,

Sandeep. v