Support Questions

Find answers, ask questions, and share your expertise

Cloudera Manager distribution error

avatar
Explorer

I am trying to install cloudera manager on single host in psuedo distribution. The OS is ubuntu 16.04. I am getting the error at distribution phase of installation. It's showing at first as connection reset by peer and then connection refused.

 

This is the log file

 

[04/Oct/2017 20:41:27 +0000] 5147 MainThread agent        INFO     Created /run/cloudera-scm-agent/process/1-cluster-host-inspector
[04/Oct/2017 20:41:27 +0000] 5147 MainThread agent        INFO     Chowning /run/cloudera-scm-agent/process/1-cluster-host-inspector to root (0) root (0)
[04/Oct/2017 20:41:27 +0000] 5147 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/process/1-cluster-host-inspector to 0751
[04/Oct/2017 20:41:27 +0000] 5147 MainThread agent        INFO     Created /run/cloudera-scm-agent/process/1-cluster-host-inspector/logs
[04/Oct/2017 20:41:27 +0000] 5147 MainThread agent        INFO     Chowning /run/cloudera-scm-agent/process/1-cluster-host-inspector/logs to root (0) root (0)
[04/Oct/2017 20:41:27 +0000] 5147 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/process/1-cluster-host-inspector/logs to 0751
[04/Oct/2017 20:41:27 +0000] 5147 MainThread process      INFO     [1-cluster-host-inspector] Refreshing process files: None
[04/Oct/2017 20:41:27 +0000] 5147 MainThread parcel       INFO     prepare_environment begin: {}, [], []
[04/Oct/2017 20:41:27 +0000] 5147 MainThread parcel       INFO     No parcels activated for use
[04/Oct/2017 20:41:27 +0000] 5147 MainThread __init__     INFO     Extracted 1 files and 0 dirs to /run/cloudera-scm-agent/process/1-cluster-host-inspector.
[04/Oct/2017 20:41:27 +0000] 5147 MainThread process      INFO     [1-cluster-host-inspector] Launching process. one-off True, command mgmt/mgmt.sh, args [u'inspector', u'input.json', u'output.json', u'DEFAULT']
[04/Oct/2017 20:41:27 +0000] 5147 MainThread agent        INFO     Triggering supervisord update.
[04/Oct/2017 20:41:27 +0000] 5147 MainThread process      INFO     Begin audit plugin refresh
[04/Oct/2017 20:41:27 +0000] 5147 MainThread navigator_plugin INFO     Scheduling a refresh for Audit Plugin for cluster-host-inspector with pipelines []
[04/Oct/2017 20:41:27 +0000] 5147 MainThread process      INFO     Begin metadata plugin refresh
[04/Oct/2017 20:41:27 +0000] 5147 MainThread navigator_plugin INFO     Scheduling a refresh for Metadata Plugin for cluster-host-inspector with pipelines []
[04/Oct/2017 20:41:27 +0000] 5147 MainThread process      INFO     Not creating a monitor for 1-cluster-host-inspector: should_monitor returns false
[04/Oct/2017 20:41:27 +0000] 5147 MainThread process      INFO     Daemon refresh complete for process 1-cluster-host-inspector.
[04/Oct/2017 20:41:30 +0000] 5147 CP Server Thread-8 _cplogging   INFO     127.0.0.1 - - [04/Oct/2017:20:41:30] "GET /process/1-cluster-host-inspector/files/inspector HTTP/1.1" 200 2252 "" "Java/1.7.0_67"
[04/Oct/2017 20:41:31 +0000] 5147 Audit-Plugin navigator_plugin INFO     Refreshing Audit Plugin for cluster-host-inspector with pipelines []
[04/Oct/2017 20:41:31 +0000] 5147 Audit-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for cluster-host-inspector (log dir: None)
[04/Oct/2017 20:41:31 +0000] 5147 Metadata-Plugin navigator_plugin INFO     Refreshing Metadata Plugin for cluster-host-inspector with pipelines []
[04/Oct/2017 20:41:31 +0000] 5147 Metadata-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for cluster-host-inspector (log dir: None)
[04/Oct/2017 20:41:45 +0000] 5147 MainThread process      INFO     [1-cluster-host-inspector] Updating process: False {u'running': (True, False), u'run_generation': (1, 2)}
[04/Oct/2017 20:41:45 +0000] 5147 MainThread process      INFO     Deactivating process 1-cluster-host-inspector
[04/Oct/2017 20:41:46 +0000] 5147 Audit-Plugin navigator_plugin INFO     stopping Audit Plugin for cluster-host-inspector with pipelines []
[04/Oct/2017 20:41:46 +0000] 5147 Audit-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for cluster-host-inspector (log dir: None)
[04/Oct/2017 20:41:46 +0000] 5147 Metadata-Plugin navigator_plugin INFO     stopping Metadata Plugin for cluster-host-inspector with pipelines []
[04/Oct/2017 20:41:46 +0000] 5147 Metadata-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for cluster-host-inspector (log dir: None)
[04/Oct/2017 20:44:50 +0000] 5147 CP Server Thread-9 _cplogging   INFO     127.0.0.1 - - [04/Oct/2017:20:44:50] "GET /heartbeat HTTP/1.1" 200 2 "" "NING/1.0"
[04/Oct/2017 20:44:50 +0000] 5147 MainThread util         INFO     Using generic audit plugin for process zookeeper-init
[04/Oct/2017 20:44:50 +0000] 5147 MainThread util         INFO     Creating metadata plugin for process zookeeper-init
[04/Oct/2017 20:44:50 +0000] 5147 MainThread util         INFO     Using specific metadata plugin for process zookeeper-init
[04/Oct/2017 20:44:50 +0000] 5147 MainThread util         INFO     Using generic metadata plugin for process zookeeper-init
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     [2-zookeeper-init] Instantiating process
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     [2-zookeeper-init] Updating process: True {}
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     First time to activate the process [2-zookeeper-init].
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     Created /run/cloudera-scm-agent/process/2-zookeeper-init
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     Chowning /run/cloudera-scm-agent/process/2-zookeeper-init to zookeeper (996) zookeeper (995)
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/process/2-zookeeper-init to 0751
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     Created /run/cloudera-scm-agent/process/2-zookeeper-init/logs
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     Chowning /run/cloudera-scm-agent/process/2-zookeeper-init/logs to zookeeper (996) zookeeper (995)
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/process/2-zookeeper-init/logs to 0751
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     [2-zookeeper-init] Refreshing process files: None
[04/Oct/2017 20:44:50 +0000] 5147 MainThread parcel       INFO     prepare_environment begin: {u'CDH': u'5.12.1-1.cdh5.12.1.p0.3'}, [u'cdh'], [u'cdh-plugin', u'zookeeper-plugin']
[04/Oct/2017 20:44:50 +0000] 5147 MainThread parcel       INFO     The following requested parcels are not available: {}
[04/Oct/2017 20:44:50 +0000] 5147 MainThread parcel       INFO     Obtained tags ['cdh', 'impala', 'sentry', 'solr', 'spark'] for parcel CDH
[04/Oct/2017 20:44:50 +0000] 5147 MainThread parcel       INFO     prepare_environment end: {'CDH': '5.12.1-1.cdh5.12.1.p0.3'}
[04/Oct/2017 20:44:50 +0000] 5147 MainThread __init__     INFO     Extracted 10 files and 0 dirs to /run/cloudera-scm-agent/process/2-zookeeper-init.
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     [2-zookeeper-init] Evaluating resource: {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': True, u'rlimits': None, u'install': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/log/zookeeper/stacks', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'zookeeper', u'user': u'zookeeper', u'mode': 493}, u'cpu': None, u'contents': None}
[04/Oct/2017 20:44:50 +0000] 5147 MainThread os_ops       INFO     Created directory /var/log/zookeeper
[04/Oct/2017 20:44:50 +0000] 5147 MainThread os_ops       INFO     Created directory /var/log/zookeeper/stacks
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     reading limits: {u'limit_memlock': None, u'limit_fds': None}
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     [2-zookeeper-init] Launching process. one-off True, command zookeeper/zk.sh, args [u'init', u'--myid=1', u'ignoreExists']
[04/Oct/2017 20:44:50 +0000] 5147 MainThread agent        INFO     Triggering supervisord update.
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     Begin audit plugin refresh
[04/Oct/2017 20:44:50 +0000] 5147 MainThread navigator_plugin INFO     Scheduling a refresh for Audit Plugin for zookeeper-init with pipelines []
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     Begin metadata plugin refresh
[04/Oct/2017 20:44:50 +0000] 5147 MainThread navigator_plugin INFO     Scheduling a refresh for Metadata Plugin for zookeeper-init with pipelines []
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     Not creating a monitor for 2-zookeeper-init: should_monitor returns false
[04/Oct/2017 20:44:50 +0000] 5147 MainThread process      INFO     Daemon refresh complete for process 2-zookeeper-init.
[04/Oct/2017 20:44:51 +0000] 5147 Audit-Plugin navigator_plugin INFO     Refreshing Audit Plugin for zookeeper-init with pipelines []
[04/Oct/2017 20:44:51 +0000] 5147 Audit-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for zookeeper-init (log dir: None)
[04/Oct/2017 20:44:51 +0000] 5147 Metadata-Plugin navigator_plugin INFO     Refreshing Metadata Plugin for zookeeper-init with pipelines []
[04/Oct/2017 20:44:51 +0000] 5147 Metadata-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for zookeeper-init (log dir: None)
[04/Oct/2017 20:45:07 +0000] 5147 MainThread process      INFO     [2-zookeeper-init] Updating process: False {u'running': (True, False), u'run_generation': (1, 2)}
[04/Oct/2017 20:45:07 +0000] 5147 MainThread process      INFO     Deactivating process 2-zookeeper-init
[04/Oct/2017 20:45:11 +0000] 5147 Audit-Plugin navigator_plugin INFO     stopping Audit Plugin for zookeeper-init with pipelines []
[04/Oct/2017 20:45:11 +0000] 5147 Audit-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for zookeeper-init (log dir: None)
[04/Oct/2017 20:45:11 +0000] 5147 Metadata-Plugin navigator_plugin INFO     stopping Metadata Plugin for zookeeper-init with pipelines []
[04/Oct/2017 20:45:11 +0000] 5147 Metadata-Plugin navigator_plugin_pipeline INFO     Stopping Navigator Plugin Pipeline '' for zookeeper-init (log dir: None)
[04/Oct/2017 20:45:22 +0000] 5147 MainThread firehoses    INFO     Reporting interval updated: 5.0 -> 60
[04/Oct/2017 20:46:20 +0000] 5147 MonitorDaemon-Reporter firehoses    INFO     Creating a connection to the SERVICEMONITOR.
[04/Oct/2017 20:46:20 +0000] 5147 MonitorDaemon-Reporter firehoses    INFO     Creating a connection to the HOSTMONITOR.
[04/Oct/2017 20:46:20 +0000] 5147 MonitorDaemon-Reporter throttling_logger ERROR    Error sending messages to firehose: mgmt-HOSTMONITOR-58d97c64d44220f47b91dc6ed632f373
Traceback (most recent call last):
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.1-py2.7.egg/cmf/monitor/firehose.py", line 116, in _send
    self._port)
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/avro-1.6.3-py2.7.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
  File "/usr/lib/python2.7/httplib.py", line 836, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 575, in create_connection
    raise err
error: [Errno 111] Connection refused
[04/Oct/2017 20:50:07 +0000] 5147 MainThread heartbeat_tracker INFO     HB stats (seconds): num:46 LIFE_MIN:0.01 min:0.00 mean:0.03 max:0.16 LIFE_MAX:0.07
[04/Oct/2017 20:51:22 +0000] 5147 MainThread throttling_logger INFO     (15 skipped) Identified java component java6 with full version JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)  for requested version 6.
[04/Oct/2017 20:51:22 +0000] 5147 MainThread throttling_logger INFO     (15 skipped) Identified java component java7 with full version JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)  for requested version 7.
[04/Oct/2017 20:57:20 +0000] 5147 MonitorDaemon-Reporter throttling_logger ERROR    (10 skipped) Error sending messages to firehose: mgmt-HOSTMONITOR-58d97c64d44220f47b91dc6ed632f373
Traceback (most recent call last):
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.1-py2.7.egg/cmf/monitor/firehose.py", line 116, in _send
    self._port)
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/avro-1.6.3-py2.7.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
  File "/usr/lib/python2.7/httplib.py", line 836, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 575, in create_connection
    raise err
error: [Errno 111] Connection refused
[04/Oct/2017 21:00:07 +0000] 5147 MainThread heartbeat_tracker INFO     HB stats (seconds): num:40 LIFE_MIN:0.01 min:0.01 mean:0.02 max:0.03 LIFE_MAX:0.07
[04/Oct/2017 21:07:20 +0000] 5147 MonitorDaemon-Reporter throttling_logger ERROR    (9 skipped) Error sending messages to firehose: mgmt-HOSTMONITOR-58d97c64d44220f47b91dc6ed632f373
[04/Oct/2017 21:10:08 +0000] 5147 MainThread heartbeat_tracker INFO     HB stats (seconds): num:40 LIFE_MIN:0.01 min:0.01 mean:0.02 max:0.03 LIFE_MAX:0.07
[04/Oct/2017 21:17:20 +0000] 5147 MonitorDaemon-Reporter throttling_logger ERROR    (9 skipped) Error sending messages to firehose: mgmt-HOSTMONITOR-58d97c64d44220f47b91dc6ed632f373
Traceback (most recent call last):
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.1-py2.7.egg/cmf/monitor/firehose.py", line 116, in _send
    self._port)
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/avro-1.6.3-py2.7.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
  File "/usr/lib/python2.7/httplib.py", line 836, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 575, in create_connection
    raise err
error: [Errno 111] Connection refused
[04/Oct/2017 21:20:08 +0000] 5147 MainThread heartbeat_tracker INFO     HB stats (seconds): num:40 LIFE_MIN:0.01 min:0.01 mean:0.02 max:0.03 LIFE_MAX:0.07
[04/Oct/2017 21:21:23 +0000] 5147 MainThread throttling_logger INFO     (14 skipped) Identified java component java6 with full version JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)  for requested version 6.
[04/Oct/2017 21:21:23 +0000] 5147 MainThread throttling_logger INFO     (14 skipped) Identified java component java7 with full version JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)  for requested version 7.
[04/Oct/2017 21:27:20 +0000] 5147 MonitorDaemon-Reporter throttling_logger ERROR    (9 skipped) Error sending messages to firehose: mgmt-HOSTMONITOR-58d97c64d44220f47b91dc6ed632f373
Traceback (most recent call last):
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.1-py2.7.egg/cmf/monitor/firehose.py", line 116, in _send
    self._port)
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/avro-1.6.3-py2.7.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
  File "/usr/lib/python2.7/httplib.py", line 836, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 575, in create_connection
    raise err
error: [Errno 111] Connection refused
[04/Oct/2017 21:30:09 +0000] 5147 MainThread heartbeat_tracker INFO     HB stats (seconds): num:40 LIFE_MIN:0.01 min:0.01 mean:0.02 max:0.04 LIFE_MAX:0.07
[04/Oct/2017 21:38:20 +0000] 5147 MonitorDaemon-Reporter throttling_logger ERROR    (10 skipped) Error sending messages to firehose: mgmt-HOSTMONITOR-58d97c64d44220f47b91dc6ed632f373
Traceback (most recent call last):
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.1-py2.7.egg/cmf/monitor/firehose.py", line 116, in _send
    self._port)
  File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/avro-1.6.3-py2.7.egg/avro/ipc.py", line 469, in __init__
    self.conn.connect()
  File "/usr/lib/python2.7/httplib.py", line 836, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 575, in create_connection
    raise err
error: [Errno 111] Connection refused
[04/Oct/2017 21:40:09 +0000] 5147 MainThread heartbeat_tracker INFO     HB stats (seconds): num:40 LIFE_MIN:0.01 min:0.00 mean:0.01 max:0.02 LIFE_MAX:0.07

The host address i am using is 127.0.1.1 with host name as my system name. I have tried it with 127.0.0.1 but it fails at intial phase heart beat not received.

10 REPLIES 10

avatar
Super Collaborator

Please attempt to start CM server again @sbommaraju Then look up /var/log/cloudera-scm-server/cloudera-scm-server.log as this should indicate potential reason for the startup failure. Also we suggest to start your own thread in the forums if you need further help with this startup error