Reply
New Contributor
Posts: 1
Registered: ‎02-19-2016

Upgrading CM to 5.7 fails with metrics_id error

Hi 

 

I am trying to upgrade Cloudera Manager from CM 5.3 to CM 5.7. Everything went smooth until it was time to start CM. that's when I am seeing following error in my cloudera-scm-server.log file although when I check the metrics table in my cm db, it has 8016 unique metric ids.

 

 

2016-04-28 01:23:52,478 WARN main:com.cloudera.cmon.components.MetricSchemaManager: Error initializing the metric schema

java.lang.IllegalStateException: Found 18 metrics with the same metric id: mem_virtual_agent_max, mem_virtual_agent_sum, mem_virtual_agent_avg, rpc_queue_time_avg_time_nodemanager_max, allocated_containers_nodemanager_min, allocated_containers_nodemanager_std_dev, jvm_heap_committed_mb_nodemanager_min, allocated_containers_nodemanager_max, containers_running_nodemanager_min, allocated_containers_nodemanager_avg, allocated_containers_nodemanager_sum, metrics_snapshot_num_ops_nodemanager_max_rate, metrics_snapshot_num_ops_nodemanager_sum, rpc_queue_time_num_ops_nodemanager_min_rate, rpc_queue_time_num_ops_nodemanager_std_dev_rate, rpc_queue_time_num_ops_nodemanager_avg_rate, rpc_queue_time_num_ops_nodemanager_max_rate, rpc_queue_time_num_ops_nodemanager_sum, .

        at com.cloudera.cmon.components.MetricSchemaManager.createCompleteSchema(MetricSchemaManager.java:356)

        at com.cloudera.cmon.components.MetricSchemaManager.access$900(MetricSchemaManager.java:49)

        at com.cloudera.cmon.components.MetricSchemaManager$2.run(MetricSchemaManager.java:219)

        at com.cloudera.cmon.components.MetricSchemaManager$2.run(MetricSchemaManager.java:161)

        at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36)

        at com.cloudera.cmf.persist.DatabaseExecutor.execTask(DatabaseExecutor.java:101)

        at com.cloudera.cmf.persist.DatabaseExecutor.execReadWriteTask(DatabaseExecutor.java:66)

        at com.cloudera.cmon.components.MetricSchemaManager.initialize(MetricSchemaManager.java:161)

        at com.cloudera.cmon.components.MetricSchemaManager.initialize(MetricSchemaManager.java:106)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:340)

        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:293)

        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1413)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)

        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)

        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)

        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)

        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)

        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)

        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)

        at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

        at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:126)

        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:108)

 

 

Highlighted
Explorer
Posts: 7
Registered: ‎07-10-2017

Re: Upgrading CM to 5.7 fails with metrics_id error

[ Edited ]

Hi CM team.

The same problem happened when our group followed the documents to upgrade CM from 5.6.0 to 5.11.1.

Is there any solution or recommendation?

Thanks very much.