Reply
Explorer
Posts: 9
Registered: ‎03-02-2015

Custom java path for specific Mapreduce job.

I have a requirement where I am compiling a mapreduce with java 1.8 where as my yarn/mapreduce is built to use java 1.7.  I need 1.8 because a library I am using can work only with 1.8.

 

Now the problem is when I run my mapreduce program it throws 

 

java.lang.UnsupportedClassVersionError: Unsupported major.minor version 52.0

 

Can you please help me run this specific mapreduce job using java 1.8.

 

Thank you all.

Posts: 1,886
Kudos: 425
Solutions: 300
Registered: ‎07-31-2013

Re: Custom java path for specific Mapreduce job.

Unless your cluster's own runtime is JDK8, you cannot run JDK8 bytecode on
the cluster. It does not matter what things are built against, but what the
runtime is.

Java compiled bytecode is backwards compatible, but not forwards
compatible. JDK8 compiled code can only run on JDK8 or JDK9+ runtimes, but
cannot on JDK7 runtime.

If your YARN NodeManagers run over JDK7, then they will apply the same
JAVA_HOME to all their spawned containers, i.e. your tasks too will run on
JDK7.

Switch your YARN service to run with JDK8 and your submitted jars will be
acceptable after that.
Explorer
Posts: 9
Registered: ‎03-02-2015

Re: Custom java path for specific Mapreduce job.

Hi Harsh,

 

I came across http://stackoverflow.com/questions/33385645/hadoop-set-custom-jdk-path-version-in-job-configuration.  Do you think this will work in any way ?

 

Thanks,

Sathish.

Announcements