Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cloudera Manager often goes down

Cloudera Manager often goes down

Explorer

All,

 

Cloudera-scm-server often goes down . following is the error message that i see  while bringing up the Cloudera Manager.

 

2014-05-20 11:03:57,933  WARN [WebServerImpl:components.SearchRepositoryManager@181] Failed to initialize search dir, deleting it: /var/lib/cloudera-scm-server/search/lucene.en..1400608940959
org.apache.lucene.index.IndexNotFoundException: no segments* file found in org.apache.lucene.store.NIOFSDirectory@/var/lib/cloudera-scm-server/search/lucene.en..1400608940959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@437024b0: files: []
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:428)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:288)
        at org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:82)
        at com.cloudera.server.web.cmf.search.LuceneSearchRepository.<init>(LuceneSearchRepository.java:106)
        at com.cloudera.server.web.cmf.search.components.SearchRepositoryManager.initializeState(SearchRepositoryManager.java:167)
        at com.cloudera.server.web.cmf.search.components.SearchRepositoryManager.<init>(SearchRepositoryManager.java:143)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:126)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:108)
        at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:280)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1003)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:907)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
        at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:467)
        at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:483)
        at org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:358)
        at org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:325)
        at org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:127)
        at javax.servlet.GenericServlet.init(GenericServlet.java:241)
        at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440)
        at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:736)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at com.cloudera.server.cmf.WebServerImpl.run(WebServerImpl.java:273)
2014-05-20 11:03:59,255 ERROR [WebServerImpl:cmf.TsqueryAutoCompleter@391] Error getting predicates
org.apache.avro.AvroRemoteException: java.net.ConnectException: Connection refused

 

2014-05-20 11:04:30,148  WARN [2143114129@scm-web-1:cmf.StatusProvider@503] Failed to get status.
java.util.concurrent.ExecutionException: org.apache.avro.AvroRemoteException: java.net.ConnectException: Connection refused

 

I have the firewall turned off . Please help

2 REPLIES 2

Re: Cloudera Manager often goes down

Explorer

I was able to resolve this issue, after fixing the configuration errors pertaining to Cloudera Management Service(mgmt). I had 2 memory related config issue on Java_Heap_Usage and nonjava_Heap_Usage which was misconfigured , that lead to mgmt service keep failing that inturn brings the cloudera manager down as scm unable to connect services SERVICE_MONITORING & HOST_MONITORING.

Re: Cloudera Manager often goes down

New Contributor

Hi I got the same problem, Unable to resolve it. Can you please be more specific what you have done to resovle this problem. Where can i change Java_Heap_Size from terminal. 

Thanks in Advance.