Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Upgrading CM to 5.7 fails with metrics_id error

Upgrading CM to 5.7 fails with metrics_id error

New Contributor

Hi 

 

I am trying to upgrade Cloudera Manager from CM 5.3 to CM 5.7. Everything went smooth until it was time to start CM. that's when I am seeing following error in my cloudera-scm-server.log file although when I check the metrics table in my cm db, it has 8016 unique metric ids.

 

 

2016-04-28 01:23:52,478 WARN main:com.cloudera.cmon.components.MetricSchemaManager: Error initializing the metric schema

java.lang.IllegalStateException: Found 18 metrics with the same metric id: mem_virtual_agent_max, mem_virtual_agent_sum, mem_virtual_agent_avg, rpc_queue_time_avg_time_nodemanager_max, allocated_containers_nodemanager_min, allocated_containers_nodemanager_std_dev, jvm_heap_committed_mb_nodemanager_min, allocated_containers_nodemanager_max, containers_running_nodemanager_min, allocated_containers_nodemanager_avg, allocated_containers_nodemanager_sum, metrics_snapshot_num_ops_nodemanager_max_rate, metrics_snapshot_num_ops_nodemanager_sum, rpc_queue_time_num_ops_nodemanager_min_rate, rpc_queue_time_num_ops_nodemanager_std_dev_rate, rpc_queue_time_num_ops_nodemanager_avg_rate, rpc_queue_time_num_ops_nodemanager_max_rate, rpc_queue_time_num_ops_nodemanager_sum, .

        at com.cloudera.cmon.components.MetricSchemaManager.createCompleteSchema(MetricSchemaManager.java:356)

        at com.cloudera.cmon.components.MetricSchemaManager.access$900(MetricSchemaManager.java:49)

        at com.cloudera.cmon.components.MetricSchemaManager$2.run(MetricSchemaManager.java:219)

        at com.cloudera.cmon.components.MetricSchemaManager$2.run(MetricSchemaManager.java:161)

        at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36)

        at com.cloudera.cmf.persist.DatabaseExecutor.execTask(DatabaseExecutor.java:101)

        at com.cloudera.cmf.persist.DatabaseExecutor.execReadWriteTask(DatabaseExecutor.java:66)

        at com.cloudera.cmon.components.MetricSchemaManager.initialize(MetricSchemaManager.java:161)

        at com.cloudera.cmon.components.MetricSchemaManager.initialize(MetricSchemaManager.java:106)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:340)

        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:293)

        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1413)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)

        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)

        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)

        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)

        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)

        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)

        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)

        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)

        at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

        at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:126)

        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:108)

 

 

1 REPLY 1

Re: Upgrading CM to 5.7 fails with metrics_id error

Explorer

Hi CM team.

The same problem happened when our group followed the documents to upgrade CM from 5.6.0 to 5.11.1.

Is there any solution or recommendation?

Thanks very much.