Support Questions

Find answers, ask questions, and share your expertise

Installation failed. Failed to receive heartbeat from agent.

avatar
Explorer

Error Message: Installation failed. Failed to receive heartbeat from agent. • Ensure that the host's hostname is configured properly. • Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules). • Ensure that ports 9000 and 9001 are not in use on the host being added. • Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details). • If Use TLS Encryption for Agentsis enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.inihas use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here. Log file: /tmp/scm_prepare_node.bK5vQRgw using SSH_CLIENT to get the SCM hostname: 10.x.y.z 38830 22 opening logging file descriptor Starting installation script... Acquiring installation lock... BEGIN flock 4 END (0) Detecting root privileges... effective UID is 0 Detecting distribution... BEGIN grep 'Ubuntu' /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS" END (0) BEGIN grep DISTRIB_CODENAME /etc/lsb-release END (0) DISTRIB_CODENAME=xenial BEGIN echo DISTRIB_CODENAME=xenial | cut -d = -f 2 xenial END (0) Detecting Cloudera Manager Server... BEGIN host -t PTR 10.x.y.z Host z.y.x.10.in-addr.arpa. not found: 3(NXDOMAIN) END (1) BEGIN which python /usr/bin/python END (0) BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' 10.x.y.z 7182 END (0) BEGIN which wget /usr/bin/wget END (0) BEGIN wget -qO- -T 1 -t 1 http://169.254.169.254/latest/meta-data/public-hostname && /bin/echo END (4) Installing package repositories... validating format of repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list installing repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list installed BEGIN apt-key add /tmp/scm_prepare_node.bK5vQRgw/archive.key OK END (0) installing priority file /tmp/scm_prepare_node.bK5vQRgw/priorities/ubuntu_xenial/cloudera.pref priority file /tmp/scm_prepare_node.bK5vQRgw/priorities/ubuntu_xenial/cloudera.pref in-stalled Refreshing package metadata... BEGIN apt-get update Hit:1 http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5 InRelease Hit:2 https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 InRe-lease Hit:3 http://repo.mysql.com/apt/ubuntu xenial InRelease Hit:4 https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5.12.0 InRe-lease Hit:5 http://in.archive.ubuntu.com/ubuntu xenial InRelease Get:6 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] Get:7 http://in.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB] Get:8 http://security.ubuntu.com/ubuntu xenial-security/main amd64 DEP-11 Metadata [60.2 kB] Get:9 http://security.ubuntu.com/ubuntu xenial-security/main DEP-11 64x64 Icons [52.0 kB] Get:10 http://in.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB] Get:11 http://in.archive.ubuntu.com/ubuntu xenial-updates/main amd64 DEP-11 Metada-ta [305 kB] Get:12 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 DEP-11 Metadata [48.8 kB] Get:13 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [64.2 kB] Get:14 http://in.archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons [206 kB] Get:15 http://in.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [171 kB] Get:16 http://in.archive.ubuntu.com/ubuntu xenial-updates/universe DEP-11 64x64 Icons [224 kB] Get:17 http://in.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 DEP-11 Metadata [5,892 B] Get:18 http://in.archive.ubuntu.com/ubuntu xenial-backports/main amd64 DEP-11 Metadata [3,324 B] Get:19 http://in.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 DEP-11 Metadata [5,136 B] Fetched 1,451 kB in 6s (241 kB/s) Reading package lists... W: http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/dists/precise-cdh5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.12.0/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) END (0) BEGIN apt-get update Hit:1 http://repo.mysql.com/apt/ubuntu xenial InRelease Hit:2 http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5 InRelease Hit:3 https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 InRe-lease Hit:4 https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5.12.0 InRe-lease Hit:5 http://security.ubuntu.com/ubuntu xenial-security InRelease Hit:6 http://in.archive.ubuntu.com/ubuntu xenial InRelease Hit:7 http://in.archive.ubuntu.com/ubuntu xenial-updates InRelease Hit:8 http://in.archive.ubuntu.com/ubuntu xenial-backports InRelease Reading package lists... W: http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/dists/precise-cdh5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.12.0/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) END (0) Installing cloudera-manager-agent package... BEGIN dpkg -l cloudera-manager-agent | grep -E '^ii[[:space:]]*cloudera-manager-agent[[:space:]]*' ii cloudera-manager-agent 5.12.0-1.cm5120.p0.120~xenial-cm5 amd64 The Cloudera Manager Agent END (0) BEGIN echo oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-agent oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons END (0) BEGIN apt-cache show cloudera-manager-agent Package: cloudera-manager-agent Source: enterprise Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 Architecture: amd64 Maintainer: Cloudera Inc. Installed-Size: 44039 Depends: libc6 (>= 2.17), libcomerr2 (>= 1.01), libgcc1 (>= 1:3.0), libgssapi-krb5-2 (>= 1.10+dfsg~), libkrb5-3 (>= 1.6.dfsg.2), libssl1.0.0 (>= 1.0.2~beta3), libstdc++6 (>= 5.2), zlib1g (>= 1:1.2.0), lsb-base, psmisc, bash, libsasl2-modules, libsasl2-modules-gssapi-mit, libxslt1.1, libsqlite3-0, openssl, libssl-dev, libfuse2, fuse-utils | fuse, rpcbind, cloudera-manager-daemons (= 5.12.0-1.cm5120.p0.120~xenial-cm5), perl, python-psycopg2, py-thon-mysqldb, apache2, iproute2 Homepage: http://www.cloudera.com Priority: extra Section: misc Filename: pool/contrib/e/enterprise/cloudera-manager-agent_5.12.0-1.cm5120.p0.120~xenial-cm5_amd64.deb Size: 11186376 SHA256: 69618949fb10d631ac301f21f62ecab0a3631eec456b6de43ef9c6a17c107498 SHA1: 6b137a4328780d56f964403ff8c56fe7577935a0 MD5sum: f7c9fc2b393b6d61563ea34ca15b6e61 Description: The Cloudera Manager Agent The Agent is deployed to machines running services managed by Cloudera Manager. Description-md5: 482a592ee859d31443f58aa3e48fe303 END (0) Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 BEGIN apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y in-stall cloudera-manager-agent Reading package lists... Building dependency tree... Reading state information... cloudera-manager-agent is already the newest version (5.12.0-1.cm5120.p0.120~xenial-cm5). 0 upgraded, 0 newly installed, 0 to remove and 347 not upgraded. END (0) remote package cloudera-manager-agent installed Installing cloudera-manager-daemons package... BEGIN dpkg -l cloudera-manager-daemons | grep -E '^ii[[:space:]]*cloudera-manager-daemons[[:space:]]*' ii cloudera-manager-daemons 5.12.0-1.cm5120.p0.120~xenial-cm5 all Provides daemons for monitoring Hadoop and related tools. END (0) BEGIN echo oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-daemons oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons END (0) BEGIN apt-cache show cloudera-manager-daemons Package: cloudera-manager-daemons Source: enterprise Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 Architecture: all Maintainer: Cloudera Inc. Installed-Size: 869404 Depends: perl Homepage: http://www.cloudera.com Priority: extra Section: misc Filename: pool/contrib/e/enterprise/cloudera-manager-daemons_5.12.0-1.cm5120.p0.120~xenial-cm5_all.deb Size: 737883268 SHA256: cf168ac13387bef54c61034e7cfef49539b1616cfb876f91db5dfe458d237780 SHA1: a18ed00de131d0c358411104fb3d886e0c211101 MD5sum: 47813b5493281f58b36ca33868229394 Description: Provides daemons for monitoring Hadoop and related tools. Provides daemons for monitoring Hadoop and related tools. Description-md5: 35bb5bc8fc2f296f791c3bda9e345340 END (0) Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 BEGIN apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y in-stall cloudera-manager-daemons Reading package lists... Building dependency tree... Reading state information... cloudera-manager-daemons is already the newest version (5.12.0-1.cm5120.p0.120~xenial-cm5). 0 upgraded, 0 newly installed, 0 to remove and 347 not upgraded. END (0) remote package cloudera-manager-daemons installed Installing Unlimited Strength Encryption policy files. Installation not requested. Step will be skipped. Configuring Cloudera Manager Agent... BEGIN grep server_host=10.x.y.z /etc/cloudera-scm-agent/config.ini server_host=10.x.y.z END (0) scm agent is already configured BEGIN sed -e 's/#\(USER=.*\)/\1/' -i /etc/default/cloudera-scm-agent END (0) Starting Cloudera Manager Agent... BEGIN /usr/sbin/service cloudera-scm-agent status * cloudera-scm-agent.service - LSB: Cloudera SCM Agent Loaded: loaded (/etc/init.d/cloudera-scm-agent; bad; vendor preset: enabled) Active: active (exited) since Thu 2017-08-24 15:47:19 IST; 58min ago Docs: man:systemd-sysv-generator(8) Process: 9473 ExecStop=/etc/init.d/cloudera-scm-agent stop (code=exited, sta-tus=0/SUCCESS) Process: 9496 ExecStart=/etc/init.d/cloudera-scm-agent start (code=exited, sta-tus=0/SUCCESS) Aug 24 15:47:17 PC355406 systemd[1]: Starting LSB: Cloudera SCM Agent... Aug 24 15:47:17 PC355406 su[9516]: Successful su for cloudera-scm by root Aug 24 15:47:17 PC355406 su[9516]: + ??? root:cloudera-scm Aug 24 15:47:17 PC355406 su[9516]: pam_unix(su:session): session opened for user cloudera-scm by (uid=0) Aug 24 15:47:19 PC355406 cloudera-scm-agent[9496]: Starting cloudera-scm-agent: * cloudera-scm-agent started Aug 24 15:47:19 PC355406 systemd[1]: Started LSB: Cloudera SCM Agent. END (0) BEGIN /usr/sbin/service cloudera-scm-agent restart END (0) agent logs: BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/' >>[24/Aug/2017 12:55:20 +0000] 1442 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent >>[24/Aug/2017 12:55:20 +0000] 1442 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent END (0) BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/' >>[24/Aug/2017 15:47:19 +0000] 9533 MainThread agent INFO Because agent not run-ning as root, all processes will run with current user. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/cgroups >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem blkio at /sys/fs/cgroup/blkio >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem memory at /sys/fs/cgroup/memory >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpu at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpuacct at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups WARNING Could not read capabilities from /proc/cgroups. error: sudo: no tty present and no askpass program speci-fied >> >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO No cgroups capabilities found >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Setting up supervisord event monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using join timeout of 0.100000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using tolerance of 60.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread kt_renewer INFO Agent wide creden-tial cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_123 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using met-rics_url_timeout_seconds of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread ntp_monitor INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread dns_names INFO Using timeout of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Created DNS monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread stacks_collection_manager INFO Us-ing max_uncompressed_file_size_bytes: 5242880 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Importing metric sche-ma from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/share/maven', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUS-TEE_SERVER_HOME': '/usr/lib/keyt>>[24/Aug/2017 15:47:19 +0000] 9533 MainThread agent INFO Because agent not running as root, all processes will run with current user. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/cgroups >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem blkio at /sys/fs/cgroup/blkio >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem memory at /sys/fs/cgroup/memory >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpu at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpuacct at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups WARNING Could not read capabilities from /proc/cgroups. error: sudo: no tty present and no askpass program speci-fied >> >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO No cgroups capabilities found >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Setting up supervisord event monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using join timeout of 0.100000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using tolerance of 60.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread kt_renewer INFO Agent wide creden-tial cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_123 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using met-rics_url_timeout_seconds of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread ntp_monitor INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread dns_names INFO Using timeout of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Created DNS monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread stacks_collection_manager INFO Us-ing max_uncompressed_file_size_bytes: 5242880 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Importing metric sche-ma from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/share/maven', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUS-TEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'SUDO': 'sudo', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO To override these varia-bles, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/process >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/flood >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor/include >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent ERROR Failed to connect to previous supervisor. >>Traceback (most recent call last): >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor >> self.configure_supervisor_clients() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients >> supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervi-sord.conf")]) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize >> Options.realize(self, *arg, **kw) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize >> self.process_config() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config >> self.process_config_file(do_usage) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file >> self.usage(str(msg)) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage >> self.exit(2) >>SystemExit: 2 >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 daemonize WARNING Stopping dae-mon. >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 agent INFO Stopping agent... rustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'SUDO': 'sudo', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO To override these varia-bles, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/process >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/flood >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor/include >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent ERROR Failed to connect to previous supervisor. >>Traceback (most recent call last): >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor >> self.configure_supervisor_clients() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients >> supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervi-sord.conf")]) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize >> Options.realize(self, *arg, **kw) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize >> self.process_config() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config >> self.process_config_file(do_usage) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file >> self.usage(str(msg)) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage >> self.exit(2) >>SystemExit: 2 >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 daemonize WARNING Stopping dae-mon. >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 agent INFO Stopping agent... END (0) end of agent logs. scm agent restarted Installation script completed successfully. all done closing logging file descriptor My /etc/hosts file is: 127.0.0.1 localhost #127.0.0.1 PC355406 10.x.y.z PC355406 # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters Kindly Provide me a solution.

23 REPLIES 23

avatar
Champion

your /etc/hosts file should be like this 

 

192.168.121.11      cluster1

also change your hostname to cluster1 

/etc/hostname 

 

 

finally if its a test enviroment either disable the firewall or change the rules accordingly . 

this should fix you the problem. 

are you configuring single or multi node cluster ?

avatar
Explorer

I am configuring for single node only. I am new to the ubuntu system so can u please tell me what is the difference between single node and multi node.

 

Also in /etc/hosts i have to remove all the configuration and just put: 

x.y.z.a  cluster1

 

127.0.0.1 is not required in /etc/hosts file ??????

 

Currently my /etc/hosts file is like this:

My /etc/hosts file is:

 

127.0.0.1         localhost

#127.0.0.1       PC355406

10.x.y.z            PC355406

 

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback

fe00::0 ip6-localnet

ff00::0 ip6-mcastprefix

ff02::1 ip6-allnodes

ff02::2 ip6-allrouters

Kindly help me in that.

avatar
Champion

single node cluster - you will have all the service under one host mostly for testing purpose . 

multi node cluster  - you will have many host each one will have their own dameon runining for example 

node1  will have master service like - namenode , YARN 

node 2 will have impala daemons 

node 3 will have hive server2 

node 4 - can acts edge nodes 

 

to make it simple for a single node cluster configure like this 

/etc/hosts 

 

any ip address like (192.168.121.14)    server1 

 

/etc/hostname will have 

server1 

 

exit the terminal or reboot the system - make sure you have your hostname configured properly by typing 

echo $hostname. 

avatar
Explorer

hi,

i have tried the solution you suggested but still i got sam error.

i have disabled iptables,

i have also put use_tls=1 in the file /etc/cloudera-scm-agent/config.inihas

 

what else i can do to solve this problem  and how to do that as i am new to this environment??

 

Thanks in advance for helping me..

 

avatar
Champion

tell me what you have under your cloudera manager ini 

in the 

 

[general ] 

server_host, server_port, listening_port, listening_hostname

 

also whats you hostname that you had configured. 

avatar
Explorer

Hi this is my ini file:

[General]

# Hostname of the CM server.

server_host=xx.yyy.zz.aaa

 

# Port that the CM server is listening on.

server_port=7182

 

## It should not normally be necessary to modify these.

# Port that the CM agent should listen on.

# listening_port=9000

 

# IP Address that the CM agent should listen on.

# listening_ip=

 

# Hostname that the CM agent reports as its hostname. If unset, will be

# obtained in code through something like this:

#

#   python -c 'import socket; \

#              print socket.getfqdn(), \

#                    socket.gethostbyname(socket.getfqdn())'

#

# listening_hostname=

 

# An alternate hostname to report as the hostname for this host in CM.

# Useful when this agent is behind a load balancer or proxy and all

# inbound communication must connect through that proxy.

# reported_hostname=

 

# Port that supervisord should listen on.

# NB: This only takes effect if supervisord is restarted.

# supervisord_port=19001

 

# Log file.  The supervisord log file will be placed into

# the same directory.  Note that if the agent is being started via the

# init.d script, /var/log/cloudera-scm-agent/cloudera-scm-agent.out will

# also have a small amount of output (from before logging is initialized).

# log_file=/var/log/cloudera-scm-agent/cloudera-scm-agent.log

 

# Persistent state directory.  Directory to store CM agent state that

# persists across instances of the agent process and system reboots.

# Particularly, the agent's UUID is stored here.

# lib_dir=/var/lib/cloudera-scm-agent

 

# Parcel directory.  Unpacked parcels will be stored in this directory.

# Downloaded parcels will be stored in <parcel_dir>/../parcel-cache

# parcel_dir=/opt/cloudera/parcels

 

# Enable supervisord event monitoring.  Used in eager heartbeating, amongst

# other things.

# enable_supervisord_events=true

 

# Maximum time to wait (in seconds) for all metric collectors to finish

# collecting data.

max_collection_wait_seconds=10.0

 

# Maximum time to wait (in seconds) when connecting to a local role's

# webserver to fetch metrics.

metrics_url_timeout_seconds=30.0

 

# Maximum time to wait (in seconds) when connecting to a local TaskTracker

# to fetch task attempt data.

task_metrics_timeout_seconds=5.0

 

# The list of non-device (nodev) filesystem types which will be monitored.

monitored_nodev_filesystem_types=nfs,nfs4,tmpfs

 

# The list of filesystem types which are considered local for monitoring purposes.

# These filesystems are combined with the other local filesystem types found in

# /proc/filesystems

local_filesystem_whitelist=ext2,ext3,ext4,xfs

 

# The largest size impala profile log bundle that this agent will serve to the

# CM server. If the CM server requests more than this amount, the bundle will

# be limited to this size. All instances of this limit being hit are logged to

# the agent log.

impala_profile_bundle_max_bytes=1073741824

 

# The largest size stacks log bundle that this agent will serve to the CM

# server. If the CM server requests more than this amount, the bundle will be

# limited to this size. All instances of this limit being hit are logged to the

# agent log.

stacks_log_bundle_max_bytes=1073741824

 

# The size to which the uncompressed portion of a stacks log can grow before it

# is rotated. The log will then be compressed during rotation.

stacks_log_max_uncompressed_file_size_bytes=5242880

 

# The orphan process directory staleness threshold. If a diretory is more stale

# than this amount of seconds, CM agent will remove it.

orphan_process_dir_staleness_threshold=5184000

 

# The orphan process directory refresh interval. The CM agent will check the

# staleness of the orphan processes config directory every this amount of

# seconds.

orphan_process_dir_refresh_interval=3600

 

# A knob to control the agent logging level. The options are listed as follows:

# 1) DEBUG (set the agent logging level to 'logging.DEBUG')

# 2) INFO (set the agent logging level to 'logging.INFO')

scm_debug=INFO

 

# The DNS resolution collecion interval in seconds. A java base test program

# will be executed with at most this frequency to collect java DNS resolution

# metrics. The test program is only executed if the associated health test,

# Host DNS Resolution, is enabled.

dns_resolution_collection_interval_seconds=60

 

# The maximum time to wait (in seconds) for the java test program to collect

# java DNS resolution metrics.

dns_resolution_collection_timeout_seconds=30

 

# The directory location in which the agent-wide kerberos credential cache

# will be created.

# agent_wide_credential_cache_location=/var/run/cloudera-scm-agent

 

[Security]

# Use TLS and certificate validation when connecting to the CM server.

use_tls=1

 

# The maximum allowed depth of the certificate chain returned by the peer.

# The default value of 9 matches the default specified in openssl's

# SSL_CTX_set_verify.

max_cert_depth=9

 

# A file of CA certificates in PEM format. The file can contain several CA

# certificates identified by

#

# -----BEGIN CERTIFICATE-----

# ... (CA certificate in base64 encoding) ...

# -----END CERTIFICATE-----

#

# sequences. Before, between, and after the certificates text is allowed which

# can be used e.g. for descriptions of the certificates.

#

# The file is loaded once, the first time an HTTPS connection is attempted. A

# restart of the agent is required to pick up changes to the file.

#

# Note that if neither verify_cert_file or verify_cert_dir is set, certificate

# verification will not be performed.

# verify_cert_file=

 

# Directory containing CA certificates in PEM format. The files each contain one

# CA certificate. The files are looked up by the CA subject name hash value,

# which must hence be available. If more than one CA certificate with the same

# name hash value exist, the extension must be different (e.g. 9d66eef0.0,

# 9d66eef0.1 etc). The search is performed in the ordering of the extension

# number, regardless of other properties of the certificates. Use the c_rehash

# utility to create the necessary links.

#

# The certificates in the directory are only looked up when required, e.g. when

# building the certificate chain or when actually performing the verification

# of a peer certificate. The contents of the directory can thus be changed

# without an agent restart.

#

# When looking up CA certificates, the verify_cert_file is first searched, then

# those in the directory. Certificate matching is done based on the subject name,

# the key identifier (if present), and the serial number as taken from the

# certificate to be verified. If these data do not match, the next certificate

# will be tried. If a first certificate matching the parameters is found, the

# verification process will be performed; no other certificates for the same

# parameters will be searched in case of failure.

#

# Note that if neither verify_cert_file or verify_cert_dir is set, certificate

# verification will not be performed.

# verify_cert_dir=

 

# PEM file containing client private key.

# client_key_file=

 

# A command to run which returns the client private key password on stdout

# client_keypw_cmd=

 

# If client_keypw_cmd isn't specified, instead a text file containing

# the client private key password can be used.

# client_keypw_file=

 

# PEM file containing client certificate.

# client_cert_file=

 

## Location of Hadoop files.  These are the CDH locations when installed by

## packages.  Unused when CDH is installed by parcels.

[Hadoop]

#cdh_crunch_home=/usr/lib/crunch

#cdh_flume_home=/usr/lib/flume-ng

#cdh_hadoop_bin=/usr/bin/hadoop

#cdh_hadoop_home=/usr/lib/hadoop

#cdh_hbase_home=/usr/lib/hbase

#cdh_hbase_indexer_home=/usr/lib/hbase-solr

#cdh_hcat_home=/usr/lib/hive-hcatalog

#cdh_hdfs_home=/usr/lib/hadoop-hdfs

#cdh_hive_home=/usr/lib/hive

#cdh_httpfs_home=/usr/lib/hadoop-httpfs

#cdh_hue_home=/usr/share/hue

#cdh_hue_plugins_home=/usr/lib/hadoop

#cdh_impala_home=/usr/lib/impala

#cdh_llama_home=/usr/lib/llama

#cdh_mr1_home=/usr/lib/hadoop-0.20-mapreduce

#cdh_mr2_home=/usr/lib/hadoop-mapreduce

#cdh_oozie_home=/usr/lib/oozie

#cdh_parquet_home=/usr/lib/parquet

#cdh_pig_home=/usr/lib/pig

#cdh_solr_home=/usr/lib/solr

#cdh_spark_home=/usr/lib/spark

#cdh_sqoop_home=/usr/lib/sqoop

#cdh_sqoop2_home=/usr/lib/sqoop2

#cdh_yarn_home=/usr/lib/hadoop-yarn

#cdh_zookeeper_home=/usr/lib/zookeeper

#hive_default_xml=/etc/hive/conf.dist/hive-default.xml

#webhcat_default_xml=/etc/hive-webhcat/conf.dist/webhcat-default.xml

#jsvc_home=/usr/libexec/bigtop-utils

#tomcat_home=/usr/lib/bigtop-tomcat

#oracle_home=/usr/share/oracle/instantclient

 

## Location of Cloudera Management Services files.

[Cloudera]

#mgmt_home=/usr/share/cmf

 

## Location of JDBC Drivers.

[JDBC]

#cloudera_mysql_connector_jar=/usr/share/java/mysql-connector-java.jar

#cloudera_oracle_connector_jar=/usr/share/java/oracle-connector-java.jar

#By default, postgres jar is found dynamically in $MGMT_HOME/lib

#cloudera_postgresql_jdbc_jar=

 

 

 

and my hostname is amit.

and /etc/hosts file has:

xx.yyy.zz.aaa       amit

 

Kindly tell me where i am doing mistake.

avatar
Explorer

Hi,

As due to unknown reasons i'm unable to post my complete error logs in my message, i've attached a link to log. Please have a look on error log.

Thank You
Amit

avatar
Champion

i see that your host file has xxx yy something 

 

my etc/hosts file contains:
xx.yyy.zz.aaa     amit

and my hostname file contains:
amit

you should have some meaning full ip add like 192.168.138.2 

 

avatar
Explorer

the ip address is 10.239.39.115

 

Is there is anything where i need to do some changes to keep things going.

 

 

Thanks,
Amit