Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cloudera Manager 5.7.0 on Redhat 7.2, services do not start up automatically

Cloudera Manager 5.7.0 on Redhat 7.2, services do not start up automatically

Explorer

I am testing Cloudera Manager Express 5.7.0 (#76 built by jenkins on 20160401-1334 git: ec0e7e69444280aa311511998bd83e8e6572f61c) on Redhat EL 7.2; fresh install of OS, Cloudera Manager, and CDH 5.7.0. Upon a fresh reboot, None of the following Cloudera Manager services start up:

 

Activity Monitor
Event Server
Host Monitor
Service Monitor

 

In previous versions of CDH/Cloudera Manager, and on previous versions of RHEL, I've never had to manually start the above services; just on the Cloudera Manager 5.7.0 / RHEL 7.2 combination. Is anyone else seeing this?

 

Thanks,
-jonathan

5 REPLIES 5

Re: Cloudera Manager 5.7.0 on Redhat 7.2, services do not start up automatically

Super Collaborator

Hi jehalter,

 

on the node where you have the listed services, can you post the contents of /var/log/cloudera-scm-agent/supervisord.log and grep -E "(FATAL|ERROR)" /var/log/cloudera-scm-{eventserver,firehose}/*.log.out

Highlighted

Re: Cloudera Manager 5.7.0 on Redhat 7.2, services do not start up automatically

Explorer

Below are the logs that you asked for. I am just providing the log entries from today in order to keep it short, and just FYI, I shut down the Hadoop cluster and rebooted all of the nodes in order to recreate the situation. The reboot happened between 10:40 - 10:42 am.

 

/var/log/cloudera-scm-agent/supervisord.log

 

2016-04-20 10:35:57,507 ERRO pool cmflistener event buffer overflowed, discarding event 94
2016-04-20 10:35:57,507 INFO stopped: 459-hive-HIVESERVER2 (exit status 143)
2016-04-20 10:35:58,030 ERRO pool cmflistener event buffer overflowed, discarding event 95
2016-04-20 10:35:58,030 INFO stopped: 460-hive-HIVEMETASTORE (exit status 143)
2016-04-20 10:35:59,490 ERRO pool cmflistener event buffer overflowed, discarding event 96
2016-04-20 10:35:59,490 INFO stopped: 461-hive-WEBHCAT (exit status 143)
2016-04-20 10:36:04,094 ERRO pool cmflistener event buffer overflowed, discarding event 97
2016-04-20 10:36:04,094 INFO stopped: 458-sqoop-SQOOP_SERVER (exit status 143)
2016-04-20 10:36:32,549 INFO spawned: '487-HBaseShutdown' with pid 29499
2016-04-20 10:36:32,554 ERRO pool cmflistener event buffer overflowed, discarding event 98
2016-04-20 10:36:32,554 INFO success: 487-HBaseShutdown entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2016-04-20 10:36:43,839 ERRO pool cmflistener event buffer overflowed, discarding event 99
2016-04-20 10:36:43,839 INFO exited: 487-HBaseShutdown (exit status 0; expected)
2016-04-20 10:37:00,595 ERRO pool cmflistener event buffer overflowed, discarding event 100
2016-04-20 10:37:00,595 INFO exited: 428-hbase-MASTER (exit status 0; expected)
2016-04-20 10:37:03,811 ERRO pool cmflistener event buffer overflowed, discarding event 101
2016-04-20 10:37:03,811 INFO stopped: 427-hbase-HBASETHRIFTSERVER (exit status 143)
2016-04-20 10:37:05,326 ERRO pool cmflistener event buffer overflowed, discarding event 102
2016-04-20 10:37:05,327 INFO stopped: 433-hbase-HBASERESTSERVER (exit status 143)
2016-04-20 10:37:19,420 ERRO pool cmflistener event buffer overflowed, discarding event 103
2016-04-20 10:37:19,420 INFO stopped: 418-hdfs-JOURNALNODE (exit status 143)
2016-04-20 10:37:34,755 ERRO pool cmflistener event buffer overflowed, discarding event 104
2016-04-20 10:37:34,755 INFO stopped: 362-zookeeper-server (exit status 143)
2016-04-20 10:38:48,281 WARN received SIGTERM indicating exit request
2016-04-20 10:38:48,282 INFO waiting for 358-cloudera-mgmt-SERVICEMONITOR, flood, 357-cloudera-mgmt-ACTIVITYMONITOR, 359-cloudera-mgmt-EVENTSERVER, 356-cloudera-mgmt-HOSTMONITOR, 355-cloudera-mgmt-ALERTPUBLISHER to die
2016-04-20 10:38:48,283 WARN received SIGHUP indicating restart request
2016-04-20 10:38:48,289 ERRO pool cmflistener event buffer overflowed, discarding event 105
2016-04-20 10:38:48,289 INFO exited: flood (terminated by SIGTERM; not expected)
2016-04-20 10:38:49,471 ERRO pool cmflistener event buffer overflowed, discarding event 106
2016-04-20 10:38:49,471 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 129; not expected)
2016-04-20 10:38:50,096 ERRO pool cmflistener event buffer overflowed, discarding event 107
2016-04-20 10:38:50,096 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 143; not expected)
2016-04-20 10:38:50,097 ERRO pool cmflistener event buffer overflowed, discarding event 108
2016-04-20 10:38:50,097 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 129; not expected)
2016-04-20 10:38:50,111 ERRO pool cmflistener event buffer overflowed, discarding event 109
2016-04-20 10:38:50,112 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 143; not expected)
2016-04-20 10:38:50,993 ERRO pool cmflistener event buffer overflowed, discarding event 110
2016-04-20 10:38:50,993 INFO stopped: 355-cloudera-mgmt-ALERTPUBLISHER (exit status 143)
2016-04-20 10:38:51,079 CRIT Supervisor running as root (no user in config file)
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/487-HBaseShutdown.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/459-hive-HIVESERVER2.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/461-hive-WEBHCAT.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/460-hive-HIVEMETASTORE.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/458-sqoop-SQOOP_SERVER.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/427-hbase-HBASETHRIFTSERVER.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/433-hbase-HBASERESTSERVER.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/428-hbase-MASTER.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/418-hdfs-JOURNALNODE.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/362-zookeeper-server.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/359-cloudera-mgmt-EVENTSERVER.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/358-cloudera-mgmt-SERVICEMONITOR.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/357-cloudera-mgmt-ACTIVITYMONITOR.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/356-cloudera-mgmt-HOSTMONITOR.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/355-cloudera-mgmt-ALERTPUBLISHER.conf" during parsing
2016-04-20 10:38:51,079 WARN Included extra file "/run/cloudera-scm-agent/supervisor/include/flood.conf" during parsing
2016-04-20 10:38:51,123 INFO RPC interface 'supervisor' initialized
2016-04-20 10:38:51,139 INFO RPC interface 'supervisor' initialized
2016-04-20 10:38:51,139 INFO supervisord started with pid 4979
2016-04-20 10:38:52,143 INFO spawned: 'cmflistener' with pid 30280
2016-04-20 10:38:52,151 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 30282
2016-04-20 10:38:52,170 INFO spawned: '460-hive-HIVEMETASTORE' with pid 30286
2016-04-20 10:38:52,172 INFO spawned: '427-hbase-HBASETHRIFTSERVER' with pid 30289
2016-04-20 10:38:52,184 INFO spawned: '418-hdfs-JOURNALNODE' with pid 30299
2016-04-20 10:38:52,186 INFO spawned: '355-cloudera-mgmt-ALERTPUBLISHER' with pid 30305
2016-04-20 10:38:52,189 INFO spawned: '461-hive-WEBHCAT' with pid 30309
2016-04-20 10:38:52,194 INFO spawned: '487-HBaseShutdown' with pid 30313
2016-04-20 10:38:52,197 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 30314
2016-04-20 10:38:52,200 INFO spawned: 'flood' with pid 30318
2016-04-20 10:38:52,203 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 30320
2016-04-20 10:38:52,217 INFO spawned: '362-zookeeper-server' with pid 30337
2016-04-20 10:38:52,221 INFO spawned: '433-hbase-HBASERESTSERVER' with pid 30348
2016-04-20 10:38:52,224 INFO spawned: '458-sqoop-SQOOP_SERVER' with pid 30352
2016-04-20 10:38:52,227 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 30357
2016-04-20 10:38:52,230 INFO spawned: '428-hbase-MASTER' with pid 30362
2016-04-20 10:38:52,242 INFO spawned: '459-hive-HIVESERVER2' with pid 30364
2016-04-20 10:38:52,243 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:38:52,245 INFO success: 487-HBaseShutdown entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2016-04-20 10:38:53,268 INFO spawned: 'cmflistener' with pid 31204
2016-04-20 10:38:53,269 INFO success: flood entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-04-20 10:38:53,307 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:38:55,831 INFO spawned: 'cmflistener' with pid 31545
2016-04-20 10:38:55,887 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:38:59,169 INFO spawned: 'cmflistener' with pid 31669
2016-04-20 10:38:59,227 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:39:00,229 INFO gave up: cmflistener entered FATAL state, too many start retries too quickly
2016-04-20 10:39:12,117 INFO exited: 458-sqoop-SQOOP_SERVER (exit status 1; not expected)
2016-04-20 10:39:13,128 INFO success: 358-cloudera-mgmt-SERVICEMONITOR entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,128 INFO success: 460-hive-HIVEMETASTORE entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,128 INFO success: 427-hbase-HBASETHRIFTSERVER entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,128 INFO success: 418-hdfs-JOURNALNODE entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,128 INFO success: 355-cloudera-mgmt-ALERTPUBLISHER entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,128 INFO success: 461-hive-WEBHCAT entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,128 INFO success: 357-cloudera-mgmt-ACTIVITYMONITOR entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,129 ERRO pool cmflistener event buffer overflowed, discarding event 121
2016-04-20 10:39:13,129 INFO success: 359-cloudera-mgmt-EVENTSERVER entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,129 ERRO pool cmflistener event buffer overflowed, discarding event 122
2016-04-20 10:39:13,129 INFO success: 362-zookeeper-server entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,129 ERRO pool cmflistener event buffer overflowed, discarding event 123
2016-04-20 10:39:13,129 INFO success: 433-hbase-HBASERESTSERVER entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,131 INFO spawned: '458-sqoop-SQOOP_SERVER' with pid 32140
2016-04-20 10:39:13,132 ERRO pool cmflistener event buffer overflowed, discarding event 124
2016-04-20 10:39:13,132 INFO success: 356-cloudera-mgmt-HOSTMONITOR entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,133 ERRO pool cmflistener event buffer overflowed, discarding event 125
2016-04-20 10:39:13,133 INFO success: 428-hbase-MASTER entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:13,133 ERRO pool cmflistener event buffer overflowed, discarding event 126
2016-04-20 10:39:13,133 INFO success: 459-hive-HIVESERVER2 entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:39:17,103 ERRO pool cmflistener event buffer overflowed, discarding event 127
2016-04-20 10:39:17,103 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:39:17,193 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 32392
2016-04-20 10:39:17,194 INFO exited: 458-sqoop-SQOOP_SERVER (exit status 1; not expected)
2016-04-20 10:39:17,766 ERRO pool cmflistener event buffer overflowed, discarding event 128
2016-04-20 10:39:17,767 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:39:18,070 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 32435
2016-04-20 10:39:19,260 INFO spawned: '458-sqoop-SQOOP_SERVER' with pid 32481
2016-04-20 10:39:19,261 ERRO pool cmflistener event buffer overflowed, discarding event 129
2016-04-20 10:39:19,261 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:39:19,349 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 32484
2016-04-20 10:39:19,367 ERRO pool cmflistener event buffer overflowed, discarding event 130
2016-04-20 10:39:19,367 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:39:19,373 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 32490
2016-04-20 10:39:25,398 INFO exited: 458-sqoop-SQOOP_SERVER (exit status 1; not expected)
2016-04-20 10:39:28,403 INFO spawned: '458-sqoop-SQOOP_SERVER' with pid 32745
2016-04-20 10:39:29,801 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:39:30,583 INFO exited: 458-sqoop-SQOOP_SERVER (exit status 1; not expected)
2016-04-20 10:39:31,585 ERRO pool cmflistener event buffer overflowed, discarding event 131
2016-04-20 10:39:31,585 INFO gave up: 458-sqoop-SQOOP_SERVER entered FATAL state, too many start retries too quickly
2016-04-20 10:39:31,586 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 445
2016-04-20 10:39:31,670 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:39:32,674 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 489
2016-04-20 10:39:33,689 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:39:34,003 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:39:35,006 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 543
2016-04-20 10:39:35,008 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 544
2016-04-20 10:39:42,775 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:39:43,895 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:39:44,899 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 645
2016-04-20 10:39:46,111 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 688
2016-04-20 10:39:46,801 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:39:46,812 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:39:49,091 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 741
2016-04-20 10:39:49,093 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 742
2016-04-20 10:39:56,089 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:39:56,915 ERRO pool cmflistener event buffer overflowed, discarding event 132
2016-04-20 10:39:56,916 INFO exited: 428-hbase-MASTER (exit status 1; not expected)
2016-04-20 10:39:57,344 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:39:59,484 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 846
2016-04-20 10:40:00,649 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 889
2016-04-20 10:40:01,472 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:40:01,477 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:40:04,761 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 939
2016-04-20 10:40:04,763 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 940
2016-04-20 10:40:10,662 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:40:11,663 ERRO pool cmflistener event buffer overflowed, discarding event 133
2016-04-20 10:40:11,663 INFO gave up: 356-cloudera-mgmt-HOSTMONITOR entered FATAL state, too many start retries too quickly
2016-04-20 10:40:12,623 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:40:13,624 ERRO pool cmflistener event buffer overflowed, discarding event 134
2016-04-20 10:40:13,624 INFO gave up: 358-cloudera-mgmt-SERVICEMONITOR entered FATAL state, too many start retries too quickly
2016-04-20 10:40:16,586 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:40:16,662 ERRO pool cmflistener event buffer overflowed, discarding event 135
2016-04-20 10:40:16,662 INFO gave up: 359-cloudera-mgmt-EVENTSERVER entered FATAL state, too many start retries too quickly
2016-04-20 10:40:16,674 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:40:16,867 ERRO pool cmflistener event buffer overflowed, discarding event 136
2016-04-20 10:40:16,867 INFO gave up: 357-cloudera-mgmt-ACTIVITYMONITOR entered FATAL state, too many start retries too quickly
2016-04-20 10:42:38,986 CRIT Supervisor running as root (no user in config file)
2016-04-20 10:42:39,036 INFO RPC interface 'supervisor' initialized
2016-04-20 10:42:39,037 INFO RPC interface 'supervisor' initialized
2016-04-20 10:42:39,039 INFO daemonizing the supervisord process
2016-04-20 10:42:39,040 INFO supervisord started with pid 2693
2016-04-20 10:42:40,042 INFO spawned: 'cmflistener' with pid 2696
2016-04-20 10:42:40,065 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:42:41,022 INFO spawned: 'flood' with pid 2779
2016-04-20 10:42:41,280 INFO spawned: 'cmflistener' with pid 2808
2016-04-20 10:42:41,305 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:42:41,519 INFO spawned: '355-cloudera-mgmt-ALERTPUBLISHER' with pid 2869
2016-04-20 10:42:41,565 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 2891
2016-04-20 10:42:41,611 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 2923
2016-04-20 10:42:41,673 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 2944
2016-04-20 10:42:41,753 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 2962
2016-04-20 10:42:42,032 INFO success: flood entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-04-20 10:42:44,272 INFO spawned: 'cmflistener' with pid 3072
2016-04-20 10:42:44,329 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:42:47,345 INFO spawned: 'cmflistener' with pid 3088
2016-04-20 10:42:47,378 INFO exited: cmflistener (exit status 1; not expected)
2016-04-20 10:42:48,379 INFO gave up: cmflistener entered FATAL state, too many start retries too quickly
2016-04-20 10:42:55,807 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:42:56,572 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:42:56,600 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:42:57,111 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 3368
2016-04-20 10:42:57,112 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:42:58,386 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 3409
2016-04-20 10:42:58,389 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 3410
2016-04-20 10:42:58,392 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 3411
2016-04-20 10:43:01,635 INFO success: 355-cloudera-mgmt-ALERTPUBLISHER entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:43:11,658 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:43:13,409 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:43:13,487 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:43:14,255 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 3587
2016-04-20 10:43:14,255 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:43:16,339 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 3631
2016-04-20 10:43:16,342 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 3632
2016-04-20 10:43:16,344 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 3633
2016-04-20 10:43:25,832 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:43:28,839 INFO spawned: '356-cloudera-mgmt-HOSTMONITOR' with pid 3769
2016-04-20 10:43:29,685 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:43:29,772 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:43:29,800 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:43:32,811 INFO spawned: '358-cloudera-mgmt-SERVICEMONITOR' with pid 3844
2016-04-20 10:43:32,813 INFO spawned: '357-cloudera-mgmt-ACTIVITYMONITOR' with pid 3845
2016-04-20 10:43:32,816 INFO spawned: '359-cloudera-mgmt-EVENTSERVER' with pid 3846
2016-04-20 10:43:40,814 INFO exited: 356-cloudera-mgmt-HOSTMONITOR (exit status 1; not expected)
2016-04-20 10:43:40,825 INFO gave up: 356-cloudera-mgmt-HOSTMONITOR entered FATAL state, too many start retries too quickly
2016-04-20 10:43:46,284 INFO exited: 359-cloudera-mgmt-EVENTSERVER (exit status 1; not expected)
2016-04-20 10:43:46,516 INFO gave up: 359-cloudera-mgmt-EVENTSERVER entered FATAL state, too many start retries too quickly
2016-04-20 10:43:46,516 INFO exited: 358-cloudera-mgmt-SERVICEMONITOR (exit status 1; not expected)
2016-04-20 10:43:46,952 INFO gave up: 358-cloudera-mgmt-SERVICEMONITOR entered FATAL state, too many start retries too quickly
2016-04-20 10:43:46,953 INFO exited: 357-cloudera-mgmt-ACTIVITYMONITOR (exit status 1; not expected)
2016-04-20 10:43:47,955 INFO gave up: 357-cloudera-mgmt-ACTIVITYMONITOR entered FATAL state, too many start retries too quickly
2016-04-20 10:52:23,897 INFO spawned: '488-cloudera-mgmt-SERVICEMONITOR' with pid 5321
2016-04-20 10:52:32,885 INFO spawned: '489-cloudera-mgmt-ACTIVITYMONITOR' with pid 5380
2016-04-20 10:52:35,533 INFO spawned: '490-cloudera-mgmt-EVENTSERVER' with pid 5422
2016-04-20 10:52:43,897 INFO success: 488-cloudera-mgmt-SERVICEMONITOR entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:52:45,276 INFO spawned: '491-cloudera-mgmt-HOSTMONITOR' with pid 5501
2016-04-20 10:52:53,809 INFO success: 489-cloudera-mgmt-ACTIVITYMONITOR entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:52:55,820 INFO success: 490-cloudera-mgmt-EVENTSERVER entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)
2016-04-20 10:53:05,537 ERRO pool cmflistener event buffer overflowed, discarding event 0
2016-04-20 10:53:05,537 INFO success: 491-cloudera-mgmt-HOSTMONITOR entered RUNNING state, process has stayed up for > than 20 seconds (startsecs)

 

grep -E "(FATAL|ERROR)" /var/log/cloudera-scm-{eventserver,firehose}/*.log.out

 

/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 00:29:44,719 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 01:00:44,014 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 01:31:44,767 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 02:02:44,018 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 02:33:44,276 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 03:04:44,554 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 03:35:44,778 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 04:06:44,039 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 04:37:44,306 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 05:08:44,638 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 05:39:44,923 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 06:10:44,208 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 06:41:44,426 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 07:12:44,711 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 07:43:44,922 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 08:14:44,149 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 08:45:44,370 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 09:16:44,606 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 09:47:44,876 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:18:44,184 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:19,259 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:33,668 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:46,791 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:40:00,914 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:40:16,568 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:42:57,078 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:13,386 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:29,743 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:46,265 ERROR com.cloudera.cmf.eventcatcher.server.EventCatcherService: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 00:29:44,594 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 01:00:44,848 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 01:31:44,764 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 02:02:44,036 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 02:33:44,293 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 03:04:44,610 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 03:35:44,886 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 04:06:44,193 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 04:37:44,455 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 05:08:44,694 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 05:39:44,967 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 06:10:44,251 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 06:41:44,556 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 07:12:44,834 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 07:43:44,148 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 08:14:44,395 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 08:45:44,678 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 09:16:44,934 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 09:47:44,223 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:18:44,518 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:19,055 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:33,988 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:46,781 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:40:00,937 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:40:16,652 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:42:56,527 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:14,203 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:29,718 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-ACTIVITYMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:46,869 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 00:29:44,749 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 01:00:44,115 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 01:31:44,729 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 02:02:43,982 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 02:33:44,254 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 03:04:44,500 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 03:35:44,793 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 04:06:44,068 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 04:37:44,317 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 05:08:44,667 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 05:39:44,933 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 06:10:44,240 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 06:41:44,568 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 07:12:44,842 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 07:43:44,126 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 08:14:44,402 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 08:45:44,665 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 09:16:44,920 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 09:47:44,193 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:18:44,426 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:17,049 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:29,751 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:42,757 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:56,073 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:40:10,643 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:42:55,739 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:11,628 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:25,812 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-HOSTMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:40,791 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 00:05:43,969 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 00:36:44,578 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 01:07:44,459 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 01:38:45,010 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 02:09:44,949 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 02:40:44,850 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 03:11:45,250 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 03:42:44,011 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 04:13:44,298 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 04:44:44,811 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 05:15:44,761 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 05:46:44,645 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 06:17:44,290 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 06:48:44,729 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 07:19:44,830 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 07:50:44,424 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 08:21:44,985 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 08:52:45,014 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 09:23:44,332 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 09:54:44,815 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:25:44,207 ERROR com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:17,601 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:31,652 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:43,882 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:39:57,323 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:40:12,604 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:42:56,586 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:13,448 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:29,620 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.
/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-bsl-c12.uncc.edu.log.out:2016-04-20 10:43:46,449 ERROR com.cloudera.cmon.firehose.Main: Could not fetch descriptor after 5 tries, exiting.

 

Thank you,

-jonathan

 

Re: Cloudera Manager 5.7.0 on Redhat 7.2, services do not start up automatically

Super Collaborator

 

This is because Cloudera Manager started a bit slower than the Management Services. While CM is initialising, it doen't start listening on port 7180 immediately. The Management Services processes made multiple attempts to connect to Cloudera Manager on port 7180 (see "Could not fetch descriptor after 5 tries, exiting.")Subsequently, because the process did not stay running more than 20 sec supervisord gave up after 3 to consequitive retries. We will have the fix in the future release. 

Re: Cloudera Manager 5.7.0 on Redhat 7.2, services do not start up automatically

Expert Contributor

Hi,

 

Is there any otehr way to fix this issue..?

Re: Cloudera Manager 5.7.0 on Redhat 7.2, services do not start up automatically

New Contributor

hi ,did you fixed this,i have the same problem.