Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

HDP 2.6.3 Install & Upgrade Issue

Explorer

I'm upgrading hdp 2.6.2.14 to 2.6.3.0, but I'm stuck in this issue and cannot finalize upgrade procedure.

JAVA ENV : jdk 1.7

To my way of thinking, 'java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0' this issue is different version between jdk7 and jdk8. Maybe YarnShuffleService.class file is compiled by jdk7 compiler by apache's some developer.

I think, this issue should be fixed ASAP. I know jdk7 is deprecated in this HDP 2.6.3.0 version and it will be removed future version.

Nodemanager Log.

2017-11-08 09:17:51,530 INFOyarn.YarnShuffleService (YarnShuffleService.java:serviceInit(156)) - Started YARN shuffle service for Spark on port 7337. Authentication is not enabled.Registered executor file is /data/log/hadoop/hadoop-yarn/nodemanager/recovery-state/nm-aux-services/spark_shuffle/registeredExecutors.ldb

2017-11-08 09:17:51,532 INFOutil.ApplicationClassLoader (ApplicationClassLoader.java:<init>(105)) - classpath: [file:/usr/hdp/2.6.3.0-235/spark2/aux/spark-2.2.0.2.6.3.0-235-yarn-shuffle.jar]

2017-11-08 09:17:51,532 INFOutil.ApplicationClassLoader (ApplicationClassLoader.java:<init>(106)) - system classes: [java., javax.accessibility., javax.activation., javax.activity., javax.annotation., javax.annotation.processing., javax.crypto., javax.imageio., javax.jws., javax.lang.model., -javax.management.j2ee., javax.management., javax.naming., javax.net., javax.print., javax.rmi., javax.script., -javax.security.auth.message., javax.security.auth., javax.security.cert., javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools., javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml., org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j., -org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml, hdfs-default.xml, mapred-default.xml, yarn-default.xml]

2017-11-08 09:17:51,534 FATAL nodemanager.NodeManager (NodeManager.java:initAndStartNodeManager(549)) - Error starting NodeManager

java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:800)

at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)

at java.net.URLClassLoader.access$100(URLClassLoader.java:71)

at java.net.URLClassLoader$1.run(URLClassLoader.java:361)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:184)

at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:165)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:274)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.java:169)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:131)

at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245)

at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:291)

at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:546)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:594)

2017-11-08 09:17:51,539 INFOimpl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping NodeManager metrics system...

2017-11-08 09:17:51,539 INFOimpl.MetricsSinkAdapter (MetricsSinkAdapter.java:publishMetricsFromQueue(141)) - timeline thread interrupted.

2017-11-08 09:17:51,540 INFOimpl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - NodeManager metrics system stopped.

2017-11-08 09:17:51,541 INFOimpl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(606)) - NodeManager metrics system shutdown complete.

2017-11-08 09:17:51,545 INFOnodemanager.NodeManager (LogAdapter.java:info(45)) - SHUTDOWN_MSG:

1 ACCEPTED SOLUTION

Super Mentor

@Peter Kim

ERROR:

java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0

.

From HDP 2.6.3 onwards JDK 1.8 Must to be used as codes are compiled agains this JDK for few components as JDK 1.7 is already EOL (End Of Life Few year back).

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.3/bk_support-matrices/content/ch_matrices-hdp...

42496-jdk-support.png

.

So you should upgrade the JDK to 1.8 and then upgrade HDP. So please Install JDK 1.8 and set the JAVA_HOME accordingly. You can also setup JDK 1.8 via ambari using:

# ambari-server setup –j /usr/lib/jvm18

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-installation/content/setup_option...

.

View solution in original post

2 REPLIES 2

Super Mentor

@Peter Kim

ERROR:

java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0

.

From HDP 2.6.3 onwards JDK 1.8 Must to be used as codes are compiled agains this JDK for few components as JDK 1.7 is already EOL (End Of Life Few year back).

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.3/bk_support-matrices/content/ch_matrices-hdp...

42496-jdk-support.png

.

So you should upgrade the JDK to 1.8 and then upgrade HDP. So please Install JDK 1.8 and set the JAVA_HOME accordingly. You can also setup JDK 1.8 via ambari using:

# ambari-server setup –j /usr/lib/jvm18

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-installation/content/setup_option...

.

Explorer

I didn't notice that actually. That because a little different version of HDP 2.6.2 to HDP 2.6.3..

And as I know, JDK 1.8 minimum requirement will be adopted HDP 3.0+....