Support Questions
Find answers, ask questions, and share your expertise

HDP 3 upgrade from 2.6 / metrics collector java class issues

Explorer

Having some issues getting the Ambari metrics HBase master process / collector process to start. Leaning towards trying to add to classpath for zookeeper? Everything else is working ok.

2018-08-20 21:32:26,397 INFO [main] zookeeper.MiniZooKeeperCluster: Started MiniZooKeeperCluster and ran successful 'stat' on client port=61181 2018-08-20 21:32:26,397 INFO [main] master.HMasterCommandLine: Starting up instance of localHBaseCluster; master=1, regionserversCount=1 2018-08-20 21:32:26,869 INFO [main] regionserver.RSRpcServices: master/nodex/192.168.x.x:0 server-side HConnection retries=350 2018-08-20 21:32:27,078 INFO [main] ipc.SimpleRpcScheduler: Using fifo as user call queue, count=3 2018-08-20 21:32:27,204 INFO [main] ipc.RpcServer: master/nodex/192.168.x.x:0: started 10 reader(s). 2018-08-20 21:32:27,276 INFO [main] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties 2018-08-20 21:32:27,325 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterorg.apache.commons.configuration2.convert.ListDelimiterHandler at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143) at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217) at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:153) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:226) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2838) Caused by: java.lang.NoClassDefFoundError: org/apache/commons/configuration2/convert/ListDelimiterHandler at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:196) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.newSink(MetricsSystemImpl.java:529) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configureSinks(MetricsSystemImpl.java:501) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:480) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:189) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:164) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50) at org.apache.hadoop.hbase.metrics.BaseSourceImpl$DefaultMetricsSystemInitializer.init(BaseSourceImpl.java:48) at org.apache.hadoop.hbase.metrics.BaseSourceImpl.<init>(BaseSourceImpl.java:71) at org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceImpl.<init>(MetricsHBaseServerSourceImpl.java:64) at org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.getSource(MetricsHBaseServerSourceFactoryImpl.java:48) at org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.create(MetricsHBaseServerSourceFactoryImpl.java:38) at org.apache.hadoop.hbase.ipc.MetricsHBaseServer.<init>(MetricsHBaseServer.java:39) at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:2013) at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:921) at org.apache.hadoop.hbase.master.MasterRpcServices.<init>(MasterRpcServices.java:248) at org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:595) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:557) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:425) at org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.<init>(HMasterCommandLine.java:309) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139) ... 7 more Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration2.convert.ListDelimiterHandler at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

1 ACCEPTED SOLUTION

@Nathan Rutland

Did you apply post installation tasks? can you run "rpm -qa | grep metrics" and see if those are pointing to 2.7 (ambari) version ? if not you will have to do https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/upgrade_ambari_me...

View solution in original post

3 REPLIES 3

@Nathan Rutland

Did you apply post installation tasks? can you run "rpm -qa | grep metrics" and see if those are pointing to 2.7 (ambari) version ? if not you will have to do https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/upgrade_ambari_me...

@Nathan Rutland thanks for confirming that it has helped. Can you please mark this as correct answer.

Explorer

@amarnath reddy pappu

I thought I updated everything on that node. The other 2 rpm's were 2.7 but the collector was 2.6. Did the yum upgrade and all is well. Thanks for your assistance. Very apprecitated. I'm going to snapshot all of the VMs now.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.