Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP 2.6.3 Install & Upgrade Issue

Solved Go to solution

HDP 2.6.3 Install & Upgrade Issue

New Contributor

I'm upgrading hdp 2.6.2.14 to 2.6.3.0, but I'm stuck in this issue and cannot finalize upgrade procedure.

JAVA ENV : jdk 1.7

To my way of thinking, 'java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0' this issue is different version between jdk7 and jdk8. Maybe YarnShuffleService.class file is compiled by jdk7 compiler by apache's some developer.

I think, this issue should be fixed ASAP. I know jdk7 is deprecated in this HDP 2.6.3.0 version and it will be removed future version.

Nodemanager Log.

2017-11-08 09:17:51,530 INFOyarn.YarnShuffleService (YarnShuffleService.java:serviceInit(156)) - Started YARN shuffle service for Spark on port 7337. Authentication is not enabled.Registered executor file is /data/log/hadoop/hadoop-yarn/nodemanager/recovery-state/nm-aux-services/spark_shuffle/registeredExecutors.ldb

2017-11-08 09:17:51,532 INFOutil.ApplicationClassLoader (ApplicationClassLoader.java:<init>(105)) - classpath: [file:/usr/hdp/2.6.3.0-235/spark2/aux/spark-2.2.0.2.6.3.0-235-yarn-shuffle.jar]

2017-11-08 09:17:51,532 INFOutil.ApplicationClassLoader (ApplicationClassLoader.java:<init>(106)) - system classes: [java., javax.accessibility., javax.activation., javax.activity., javax.annotation., javax.annotation.processing., javax.crypto., javax.imageio., javax.jws., javax.lang.model., -javax.management.j2ee., javax.management., javax.naming., javax.net., javax.print., javax.rmi., javax.script., -javax.security.auth.message., javax.security.auth., javax.security.cert., javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools., javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml., org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j., -org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml, hdfs-default.xml, mapred-default.xml, yarn-default.xml]

2017-11-08 09:17:51,534 FATAL nodemanager.NodeManager (NodeManager.java:initAndStartNodeManager(549)) - Error starting NodeManager

java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:800)

at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)

at java.net.URLClassLoader.access$100(URLClassLoader.java:71)

at java.net.URLClassLoader$1.run(URLClassLoader.java:361)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:184)

at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:165)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:274)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.java:169)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:131)

at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245)

at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:291)

at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:546)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:594)

2017-11-08 09:17:51,539 INFOimpl.MetricsSystemImpl (MetricsSystemImpl.java:stop(211)) - Stopping NodeManager metrics system...

2017-11-08 09:17:51,539 INFOimpl.MetricsSinkAdapter (MetricsSinkAdapter.java:publishMetricsFromQueue(141)) - timeline thread interrupted.

2017-11-08 09:17:51,540 INFOimpl.MetricsSystemImpl (MetricsSystemImpl.java:stop(217)) - NodeManager metrics system stopped.

2017-11-08 09:17:51,541 INFOimpl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(606)) - NodeManager metrics system shutdown complete.

2017-11-08 09:17:51,545 INFOnodemanager.NodeManager (LogAdapter.java:info(45)) - SHUTDOWN_MSG:

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDP 2.6.3 Install & Upgrade Issue

Super Mentor

@Peter Kim

ERROR:

java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0

.

From HDP 2.6.3 onwards JDK 1.8 Must to be used as codes are compiled agains this JDK for few components as JDK 1.7 is already EOL (End Of Life Few year back).

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.3/bk_support-matrices/content/ch_matrices-hdp...

42496-jdk-support.png

.

So you should upgrade the JDK to 1.8 and then upgrade HDP. So please Install JDK 1.8 and set the JAVA_HOME accordingly. You can also setup JDK 1.8 via ambari using:

# ambari-server setup –j /usr/lib/jvm18

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-installation/content/setup_option...

.

2 REPLIES 2

Re: HDP 2.6.3 Install & Upgrade Issue

Super Mentor

@Peter Kim

ERROR:

java.lang.UnsupportedClassVersionError: org/apache/spark/network/yarn/YarnShuffleService : Unsupported major.minor version 52.0

.

From HDP 2.6.3 onwards JDK 1.8 Must to be used as codes are compiled agains this JDK for few components as JDK 1.7 is already EOL (End Of Life Few year back).

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.3/bk_support-matrices/content/ch_matrices-hdp...

42496-jdk-support.png

.

So you should upgrade the JDK to 1.8 and then upgrade HDP. So please Install JDK 1.8 and set the JAVA_HOME accordingly. You can also setup JDK 1.8 via ambari using:

# ambari-server setup –j /usr/lib/jvm18

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-installation/content/setup_option...

.

Re: HDP 2.6.3 Install & Upgrade Issue

New Contributor

I didn't notice that actually. That because a little different version of HDP 2.6.2 to HDP 2.6.3..

And as I know, JDK 1.8 minimum requirement will be adopted HDP 3.0+....