Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cloudera Manager not starting

Highlighted

Cloudera Manager not starting

Explorer

Cloudera Manager stop working.

Below is the log from  cloudera-scm-server.log

 

2015-08-12 14:06:08,123 INFO main:com.cloudera.cmf.scheduler.CmfScheduler: Added command schedule '1' to the scheduler
2015-08-12 14:06:08,224 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
2015-08-12 14:06:08,224 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
2015-08-12 14:06:08,224 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
2015-08-12 14:06:08,224 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
2015-08-12 14:06:08,224 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.properties
2015-08-12 14:06:08,225 INFO main:com.cloudera.server.cmf.components.EmbeddedDbManager: Reading embedded database password from /var/lib/cloudera-scm-server-db/data/generated_password.txt
2015-08-12 14:06:08,225 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.properties
2015-08-12 14:06:08,256 INFO main:com.cloudera.cmf.cdhclient.util.CDHUrlClassLoader: Detected that this program is running in a JAVA 1.7.0_67 JVM. CDH5 jars will be loaded from:lib/cdh5
2015-08-12 14:06:08,256 INFO main:com.cloudera.cmf.cdhclient.CdhExecutorFactory: Adding options to bad options list:

 

 

I tried from curl

 

curl x.x.x.x:7180
curl: (7) couldn't connect to host

 

 

netstat return none.

 

netstat -anp | grep 7180

 

 

 

Any help is highly appreciated.

 

4 REPLIES 4

Re: Cloudera Manager not starting

Super Guru

Hello,

 

The cloudera-scm-server.log file is the right place to look for issues, but I think the snippet you provided does not contain the most relevant lines.  Can you look for ERROR or Exception in the log?  Is it possilbe that the server was still starting when you were trying to access it?  CM can take a minute or two.  Try grepping for "718" in the log to see if it started.  (7180 is the non-ssl port, 7183 is the TLS port if you enabled it, 7182 is the agent port).

 

Regards,

 

Ben

Re: Cloudera Manager not starting

Explorer

Hi Ben,

 

Thanks for the reply.

 

Below is the output from grepping 7180

 

 cat /var/log/cloudera-scm-server/cloudera-scm-server.log | grep 718
2015-08-12 14:05:54,718 INFO main:com.cloudera.enterprise.CommonMain: Statistics not enabled, JMX will not be registered
2015-08-12 14:06:06,718 INFO main:com.cloudera.enterprise.Translator: Loading bundle 'actionItem' from 'actionItem_pt_PT.properties'
2015-08-12 14:06:06,718 INFO main:com.cloudera.enterprise.Translator: Loading bundle 'actionItem' from 'actionItem_pt_BR.properties'
2015-08-12 14:06:06,718 INFO main:com.cloudera.enterprise.Translator: Loading bundle 'actionItem' from 'actionItem_ja.properties'
2015-08-12 14:06:06,718 INFO main:com.cloudera.enterprise.Translator: Loading bundle 'actionItem' from 'actionItem_ko.properties'

 

Also, below is the cloudera-scm-server.out

 

JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
java.lang.OutOfMemoryError: Java heap space
Dumping heap to /tmp/java_pid1264.hprof ...
Heap dump file created [2011978655 bytes in 4.520 secs]
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill -9 %p"
# Executing /bin/sh -c "kill -9 1264"...
Killed (core dumped)

 

 

I did try to access the portal even after 30 mins , but still it didnt help.

 

Let me know if i have provided the correct information. 

 

Re: Cloudera Manager not starting

Super Guru

Good call looking in the cloudera-scm-server.out file.

 

As we see that heap was exhausted, assuming this is not a memory leak or something of the sort, Cloudera Manager may need more heap to operate.  This can be configured in:

 

   /etc/default/cloudera-scm-server

 

You could, for instance, change "-Xmx2G" to "-Xmx3G" or "-Xmx4G"

 

If the problem still happens, perhaps the heap dumps will yeild some clues.

 

- Ben

Re: Cloudera Manager not starting

Explorer

Changed from 2G to 4G

 

export CMF_JAVA_OPTS="-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"

 

Still error from cloudera-scm-server.out

 

JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
java.lang.OutOfMemoryError: Java heap space
Dumping heap to /tmp/java_pid30831.hprof ...
Heap dump file created [4141312052 bytes in 11.785 secs]
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill -9 %p"
# Executing /bin/sh -c "kill -9 30831"...
Killed (core dumped)