Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Updating HDP 2.4 to 2.6 storm nimbus not starting.

avatar
Contributor

I upgraded from HDP 2.4 to 2.6 using express upgrade. All services are running fine except Nimbus. I get following error:

java.lang.NoClassDefFoundError: backtype/storm/metric/IClusterReporter
        at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_40]
        at java.lang.ClassLoader.defineClass(ClassLoader.java:760) ~[?:1.8.0_40]
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_40]
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) ~[?:1.8.0_40]
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73) ~[?:1.8.0_40]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368) ~[?:1.8.0_40]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362) ~[?:1.8.0_40]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_40]
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361) ~[?:1.8.0_40]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_40]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_40]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_40]
        at java.lang.Class.forName0(Native Method) ~[?:1.8.0_40]
        at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_40]
        at org.apache.storm.metric.ClusterMetricsConsumerExecutor.prepare(ClusterMetricsConsumerExecutor.java:45) ~[storm-core-1.1.0.2.6.2.0-205.jar:1.1.0.2.6.2.0-205]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_40]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_40]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_40]
        at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_40]
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) ~[clojure-1.7.0.jar:?]
        at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:313) ~[clojure-1.7.0.jar:?]
        at org.apache.storm.daemon.nimbus$fn__9790$exec_fn__3654__auto____9791.invoke(nimbus.clj:2469) ~[storm-core-1.1.0.2.6.2.0-205.jar:1.1.0.2.6.2.0-205]
        at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]
        at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
        at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
        at org.apache.storm.daemon.nimbus$fn__9790$service_handler__9823.doInvoke(nimbus.clj:2446) ~[storm-core-1.1.0.2.6.2.0-205.jar:1.1.0.2.6.2.0-205]
        at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]
        at org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2534) ~[storm-core-1.1.0.2.6.2.0-205.jar:1.1.0.2.6.2.0-205]
        at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2567) ~[storm-core-1.1.0.2.6.2.0-205.jar:1.1.0.2.6.2.0-205]
        at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2590) ~[storm-core-1.1.0.2.6.2.0-205.jar:1.1.0.2.6.2.0-205]
        at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]
        at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
        at org.apache.storm.daemon.nimbus.main(Unknown Source) ~[storm-core-1.1.0.2.6.2.0-205.jar:1.1.0.2.6.2.0-205]
Caused by: java.lang.ClassNotFoundException: backtype.storm.metric.IClusterReporter
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_40]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_40]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_40]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_40]
        ... 33 more
1 ACCEPTED SOLUTION

avatar
@Eon kitex

There is an ambari metrics plugin used by Nimbus that also needed to be upgrade and was provided in the ambari-metrics-monitor or hadoop-sink components.

To perform the upgrade from both ambari-metrics-monitor and ambari-metrics-hadoop-sink on all nodes of the cluster.

If you have metrics service added, it is recommended to shutdown all of the Ambari Metrics components and run the following:

-> yum upgrade ambari-metrics-monitor
-> yum upgrade ambari-metrics-hadoop-sink

on each node of the cluster and then try starting Nimbus service.

View solution in original post

2 REPLIES 2

avatar
@Eon kitex

There is an ambari metrics plugin used by Nimbus that also needed to be upgrade and was provided in the ambari-metrics-monitor or hadoop-sink components.

To perform the upgrade from both ambari-metrics-monitor and ambari-metrics-hadoop-sink on all nodes of the cluster.

If you have metrics service added, it is recommended to shutdown all of the Ambari Metrics components and run the following:

-> yum upgrade ambari-metrics-monitor
-> yum upgrade ambari-metrics-hadoop-sink

on each node of the cluster and then try starting Nimbus service.

avatar
Contributor

I had done this in one of the node but forgot to perform this on all of the nodes. It works seemlessly.