Support Questions
Find answers, ask questions, and share your expertise

HDP 3 upgrade from 2.6 / metrics collector java class issues

Explorer

Having some issues getting the Ambari metrics HBase master process / collector process to start. Leaning towards trying to add to classpath for zookeeper? Everything else is working ok.

2018-08-20 21:32:26,397 INFO [main] zookeeper.MiniZooKeeperCluster: Started MiniZooKeeperCluster and ran successful 'stat' on client port=61181 2018-08-20 21:32:26,397 INFO [main] master.HMasterCommandLine: Starting up instance of localHBaseCluster; master=1, regionserversCount=1 2018-08-20 21:32:26,869 INFO [main] regionserver.RSRpcServices: master/nodex/192.168.x.x:0 server-side HConnection retries=350 2018-08-20 21:32:27,078 INFO [main] ipc.SimpleRpcScheduler: Using fifo as user call queue, count=3 2018-08-20 21:32:27,204 INFO [main] ipc.RpcServer: master/nodex/192.168.x.x:0: started 10 reader(s). 2018-08-20 21:32:27,276 INFO [main] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties 2018-08-20 21:32:27,325 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterorg.apache.commons.configuration2.convert.ListDelimiterHandler at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143) at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217) at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:153) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:226) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2838) Caused by: java.lang.NoClassDefFoundError: org/apache/commons/configuration2/convert/ListDelimiterHandler at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:196) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.newSink(MetricsSystemImpl.java:529) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configureSinks(MetricsSystemImpl.java:501) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:480) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:189) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:164) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50) at org.apache.hadoop.hbase.metrics.BaseSourceImpl$DefaultMetricsSystemInitializer.init(BaseSourceImpl.java:48) at org.apache.hadoop.hbase.metrics.BaseSourceImpl.<init>(BaseSourceImpl.java:71) at org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceImpl.<init>(MetricsHBaseServerSourceImpl.java:64) at org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.getSource(MetricsHBaseServerSourceFactoryImpl.java:48) at org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.create(MetricsHBaseServerSourceFactoryImpl.java:38) at org.apache.hadoop.hbase.ipc.MetricsHBaseServer.<init>(MetricsHBaseServer.java:39) at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:2013) at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:921) at org.apache.hadoop.hbase.master.MasterRpcServices.<init>(MasterRpcServices.java:248) at org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:595) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:557) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:425) at org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.<init>(HMasterCommandLine.java:309) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139) ... 7 more Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration2.convert.ListDelimiterHandler at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDP 3 upgrade from 2.6 / metrics collector java class issues

@Nathan Rutland

Did you apply post installation tasks? can you run "rpm -qa | grep metrics" and see if those are pointing to 2.7 (ambari) version ? if not you will have to do https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/upgrade_ambari_me...

View solution in original post

3 REPLIES 3

Re: HDP 3 upgrade from 2.6 / metrics collector java class issues

@Nathan Rutland

Did you apply post installation tasks? can you run "rpm -qa | grep metrics" and see if those are pointing to 2.7 (ambari) version ? if not you will have to do https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-upgrade/content/upgrade_ambari_me...

View solution in original post

Re: HDP 3 upgrade from 2.6 / metrics collector java class issues

@Nathan Rutland thanks for confirming that it has helped. Can you please mark this as correct answer.

Re: HDP 3 upgrade from 2.6 / metrics collector java class issues

Explorer

@amarnath reddy pappu

I thought I updated everything on that node. The other 2 rpm's were 2.7 but the collector was 2.6. Did the yum upgrade and all is well. Thanks for your assistance. Very apprecitated. I'm going to snapshot all of the VMs now.