Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Installation failed. Failed to receive heartbeat from agent.

avatar
Explorer

Error Message: Installation failed. Failed to receive heartbeat from agent. • Ensure that the host's hostname is configured properly. • Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules). • Ensure that ports 9000 and 9001 are not in use on the host being added. • Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details). • If Use TLS Encryption for Agentsis enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.inihas use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here. Log file: /tmp/scm_prepare_node.bK5vQRgw using SSH_CLIENT to get the SCM hostname: 10.x.y.z 38830 22 opening logging file descriptor Starting installation script... Acquiring installation lock... BEGIN flock 4 END (0) Detecting root privileges... effective UID is 0 Detecting distribution... BEGIN grep 'Ubuntu' /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS" END (0) BEGIN grep DISTRIB_CODENAME /etc/lsb-release END (0) DISTRIB_CODENAME=xenial BEGIN echo DISTRIB_CODENAME=xenial | cut -d = -f 2 xenial END (0) Detecting Cloudera Manager Server... BEGIN host -t PTR 10.x.y.z Host z.y.x.10.in-addr.arpa. not found: 3(NXDOMAIN) END (1) BEGIN which python /usr/bin/python END (0) BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' 10.x.y.z 7182 END (0) BEGIN which wget /usr/bin/wget END (0) BEGIN wget -qO- -T 1 -t 1 http://169.254.169.254/latest/meta-data/public-hostname && /bin/echo END (4) Installing package repositories... validating format of repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list installing repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list installed BEGIN apt-key add /tmp/scm_prepare_node.bK5vQRgw/archive.key OK END (0) installing priority file /tmp/scm_prepare_node.bK5vQRgw/priorities/ubuntu_xenial/cloudera.pref priority file /tmp/scm_prepare_node.bK5vQRgw/priorities/ubuntu_xenial/cloudera.pref in-stalled Refreshing package metadata... BEGIN apt-get update Hit:1 http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5 InRelease Hit:2 https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 InRe-lease Hit:3 http://repo.mysql.com/apt/ubuntu xenial InRelease Hit:4 https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5.12.0 InRe-lease Hit:5 http://in.archive.ubuntu.com/ubuntu xenial InRelease Get:6 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] Get:7 http://in.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB] Get:8 http://security.ubuntu.com/ubuntu xenial-security/main amd64 DEP-11 Metadata [60.2 kB] Get:9 http://security.ubuntu.com/ubuntu xenial-security/main DEP-11 64x64 Icons [52.0 kB] Get:10 http://in.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB] Get:11 http://in.archive.ubuntu.com/ubuntu xenial-updates/main amd64 DEP-11 Metada-ta [305 kB] Get:12 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 DEP-11 Metadata [48.8 kB] Get:13 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [64.2 kB] Get:14 http://in.archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons [206 kB] Get:15 http://in.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [171 kB] Get:16 http://in.archive.ubuntu.com/ubuntu xenial-updates/universe DEP-11 64x64 Icons [224 kB] Get:17 http://in.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 DEP-11 Metadata [5,892 B] Get:18 http://in.archive.ubuntu.com/ubuntu xenial-backports/main amd64 DEP-11 Metadata [3,324 B] Get:19 http://in.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 DEP-11 Metadata [5,136 B] Fetched 1,451 kB in 6s (241 kB/s) Reading package lists... W: http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/dists/precise-cdh5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.12.0/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) END (0) BEGIN apt-get update Hit:1 http://repo.mysql.com/apt/ubuntu xenial InRelease Hit:2 http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5 InRelease Hit:3 https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 InRe-lease Hit:4 https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5.12.0 InRe-lease Hit:5 http://security.ubuntu.com/ubuntu xenial-security InRelease Hit:6 http://in.archive.ubuntu.com/ubuntu xenial InRelease Hit:7 http://in.archive.ubuntu.com/ubuntu xenial-updates InRelease Hit:8 http://in.archive.ubuntu.com/ubuntu xenial-backports InRelease Reading package lists... W: http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/dists/precise-cdh5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.12.0/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) END (0) Installing cloudera-manager-agent package... BEGIN dpkg -l cloudera-manager-agent | grep -E '^ii[[:space:]]*cloudera-manager-agent[[:space:]]*' ii cloudera-manager-agent 5.12.0-1.cm5120.p0.120~xenial-cm5 amd64 The Cloudera Manager Agent END (0) BEGIN echo oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-agent oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons END (0) BEGIN apt-cache show cloudera-manager-agent Package: cloudera-manager-agent Source: enterprise Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 Architecture: amd64 Maintainer: Cloudera Inc. Installed-Size: 44039 Depends: libc6 (>= 2.17), libcomerr2 (>= 1.01), libgcc1 (>= 1:3.0), libgssapi-krb5-2 (>= 1.10+dfsg~), libkrb5-3 (>= 1.6.dfsg.2), libssl1.0.0 (>= 1.0.2~beta3), libstdc++6 (>= 5.2), zlib1g (>= 1:1.2.0), lsb-base, psmisc, bash, libsasl2-modules, libsasl2-modules-gssapi-mit, libxslt1.1, libsqlite3-0, openssl, libssl-dev, libfuse2, fuse-utils | fuse, rpcbind, cloudera-manager-daemons (= 5.12.0-1.cm5120.p0.120~xenial-cm5), perl, python-psycopg2, py-thon-mysqldb, apache2, iproute2 Homepage: http://www.cloudera.com Priority: extra Section: misc Filename: pool/contrib/e/enterprise/cloudera-manager-agent_5.12.0-1.cm5120.p0.120~xenial-cm5_amd64.deb Size: 11186376 SHA256: 69618949fb10d631ac301f21f62ecab0a3631eec456b6de43ef9c6a17c107498 SHA1: 6b137a4328780d56f964403ff8c56fe7577935a0 MD5sum: f7c9fc2b393b6d61563ea34ca15b6e61 Description: The Cloudera Manager Agent The Agent is deployed to machines running services managed by Cloudera Manager. Description-md5: 482a592ee859d31443f58aa3e48fe303 END (0) Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 BEGIN apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y in-stall cloudera-manager-agent Reading package lists... Building dependency tree... Reading state information... cloudera-manager-agent is already the newest version (5.12.0-1.cm5120.p0.120~xenial-cm5). 0 upgraded, 0 newly installed, 0 to remove and 347 not upgraded. END (0) remote package cloudera-manager-agent installed Installing cloudera-manager-daemons package... BEGIN dpkg -l cloudera-manager-daemons | grep -E '^ii[[:space:]]*cloudera-manager-daemons[[:space:]]*' ii cloudera-manager-daemons 5.12.0-1.cm5120.p0.120~xenial-cm5 all Provides daemons for monitoring Hadoop and related tools. END (0) BEGIN echo oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-daemons oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons END (0) BEGIN apt-cache show cloudera-manager-daemons Package: cloudera-manager-daemons Source: enterprise Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 Architecture: all Maintainer: Cloudera Inc. Installed-Size: 869404 Depends: perl Homepage: http://www.cloudera.com Priority: extra Section: misc Filename: pool/contrib/e/enterprise/cloudera-manager-daemons_5.12.0-1.cm5120.p0.120~xenial-cm5_all.deb Size: 737883268 SHA256: cf168ac13387bef54c61034e7cfef49539b1616cfb876f91db5dfe458d237780 SHA1: a18ed00de131d0c358411104fb3d886e0c211101 MD5sum: 47813b5493281f58b36ca33868229394 Description: Provides daemons for monitoring Hadoop and related tools. Provides daemons for monitoring Hadoop and related tools. Description-md5: 35bb5bc8fc2f296f791c3bda9e345340 END (0) Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 BEGIN apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y in-stall cloudera-manager-daemons Reading package lists... Building dependency tree... Reading state information... cloudera-manager-daemons is already the newest version (5.12.0-1.cm5120.p0.120~xenial-cm5). 0 upgraded, 0 newly installed, 0 to remove and 347 not upgraded. END (0) remote package cloudera-manager-daemons installed Installing Unlimited Strength Encryption policy files. Installation not requested. Step will be skipped. Configuring Cloudera Manager Agent... BEGIN grep server_host=10.x.y.z /etc/cloudera-scm-agent/config.ini server_host=10.x.y.z END (0) scm agent is already configured BEGIN sed -e 's/#\(USER=.*\)/\1/' -i /etc/default/cloudera-scm-agent END (0) Starting Cloudera Manager Agent... BEGIN /usr/sbin/service cloudera-scm-agent status * cloudera-scm-agent.service - LSB: Cloudera SCM Agent Loaded: loaded (/etc/init.d/cloudera-scm-agent; bad; vendor preset: enabled) Active: active (exited) since Thu 2017-08-24 15:47:19 IST; 58min ago Docs: man:systemd-sysv-generator(8) Process: 9473 ExecStop=/etc/init.d/cloudera-scm-agent stop (code=exited, sta-tus=0/SUCCESS) Process: 9496 ExecStart=/etc/init.d/cloudera-scm-agent start (code=exited, sta-tus=0/SUCCESS) Aug 24 15:47:17 PC355406 systemd[1]: Starting LSB: Cloudera SCM Agent... Aug 24 15:47:17 PC355406 su[9516]: Successful su for cloudera-scm by root Aug 24 15:47:17 PC355406 su[9516]: + ??? root:cloudera-scm Aug 24 15:47:17 PC355406 su[9516]: pam_unix(su:session): session opened for user cloudera-scm by (uid=0) Aug 24 15:47:19 PC355406 cloudera-scm-agent[9496]: Starting cloudera-scm-agent: * cloudera-scm-agent started Aug 24 15:47:19 PC355406 systemd[1]: Started LSB: Cloudera SCM Agent. END (0) BEGIN /usr/sbin/service cloudera-scm-agent restart END (0) agent logs: BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/' >>[24/Aug/2017 12:55:20 +0000] 1442 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent >>[24/Aug/2017 12:55:20 +0000] 1442 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent END (0) BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/' >>[24/Aug/2017 15:47:19 +0000] 9533 MainThread agent INFO Because agent not run-ning as root, all processes will run with current user. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/cgroups >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem blkio at /sys/fs/cgroup/blkio >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem memory at /sys/fs/cgroup/memory >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpu at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpuacct at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups WARNING Could not read capabilities from /proc/cgroups. error: sudo: no tty present and no askpass program speci-fied >> >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO No cgroups capabilities found >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Setting up supervisord event monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using join timeout of 0.100000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using tolerance of 60.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread kt_renewer INFO Agent wide creden-tial cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_123 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using met-rics_url_timeout_seconds of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread ntp_monitor INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread dns_names INFO Using timeout of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Created DNS monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread stacks_collection_manager INFO Us-ing max_uncompressed_file_size_bytes: 5242880 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Importing metric sche-ma from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/share/maven', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUS-TEE_SERVER_HOME': '/usr/lib/keyt>>[24/Aug/2017 15:47:19 +0000] 9533 MainThread agent INFO Because agent not running as root, all processes will run with current user. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/cgroups >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem blkio at /sys/fs/cgroup/blkio >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem memory at /sys/fs/cgroup/memory >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpu at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpuacct at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups WARNING Could not read capabilities from /proc/cgroups. error: sudo: no tty present and no askpass program speci-fied >> >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO No cgroups capabilities found >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Setting up supervisord event monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using join timeout of 0.100000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using tolerance of 60.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread kt_renewer INFO Agent wide creden-tial cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_123 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using met-rics_url_timeout_seconds of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread ntp_monitor INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread dns_names INFO Using timeout of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Created DNS monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread stacks_collection_manager INFO Us-ing max_uncompressed_file_size_bytes: 5242880 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Importing metric sche-ma from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/share/maven', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUS-TEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'SUDO': 'sudo', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO To override these varia-bles, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/process >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/flood >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor/include >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent ERROR Failed to connect to previous supervisor. >>Traceback (most recent call last): >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor >> self.configure_supervisor_clients() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients >> supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervi-sord.conf")]) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize >> Options.realize(self, *arg, **kw) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize >> self.process_config() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config >> self.process_config_file(do_usage) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file >> self.usage(str(msg)) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage >> self.exit(2) >>SystemExit: 2 >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 daemonize WARNING Stopping dae-mon. >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 agent INFO Stopping agent... rustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'SUDO': 'sudo', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO To override these varia-bles, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/process >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/flood >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor/include >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent ERROR Failed to connect to previous supervisor. >>Traceback (most recent call last): >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor >> self.configure_supervisor_clients() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients >> supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervi-sord.conf")]) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize >> Options.realize(self, *arg, **kw) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize >> self.process_config() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config >> self.process_config_file(do_usage) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file >> self.usage(str(msg)) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage >> self.exit(2) >>SystemExit: 2 >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 daemonize WARNING Stopping dae-mon. >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 agent INFO Stopping agent... END (0) end of agent logs. scm agent restarted Installation script completed successfully. all done closing logging file descriptor My /etc/hosts file is: 127.0.0.1 localhost #127.0.0.1 PC355406 10.x.y.z PC355406 # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters Kindly Provide me a solution.

23 REPLIES 23

avatar
Champion

your /etc/hosts file should be like this 

 

192.168.121.11      cluster1

also change your hostname to cluster1 

/etc/hostname 

 

 

finally if its a test enviroment either disable the firewall or change the rules accordingly . 

this should fix you the problem. 

are you configuring single or multi node cluster ?

avatar
Explorer

I am configuring for single node only. I am new to the ubuntu system so can u please tell me what is the difference between single node and multi node.

 

Also in /etc/hosts i have to remove all the configuration and just put: 

x.y.z.a  cluster1

 

127.0.0.1 is not required in /etc/hosts file ??????

 

Currently my /etc/hosts file is like this:

My /etc/hosts file is:

 

127.0.0.1         localhost

#127.0.0.1       PC355406

10.x.y.z            PC355406

 

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback

fe00::0 ip6-localnet

ff00::0 ip6-mcastprefix

ff02::1 ip6-allnodes

ff02::2 ip6-allrouters

Kindly help me in that.

avatar
Champion

single node cluster - you will have all the service under one host mostly for testing purpose . 

multi node cluster  - you will have many host each one will have their own dameon runining for example 

node1  will have master service like - namenode , YARN 

node 2 will have impala daemons 

node 3 will have hive server2 

node 4 - can acts edge nodes 

 

to make it simple for a single node cluster configure like this 

/etc/hosts 

 

any ip address like (192.168.121.14)    server1 

 

/etc/hostname will have 

server1 

 

exit the terminal or reboot the system - make sure you have your hostname configured properly by typing 

echo $hostname. 

avatar
Explorer

hi,

i have tried the solution you suggested but still i got sam error.

i have disabled iptables,

i have also put use_tls=1 in the file /etc/cloudera-scm-agent/config.inihas

 

what else i can do to solve this problem  and how to do that as i am new to this environment??

 

Thanks in advance for helping me..

 

avatar
Champion

tell me what you have under your cloudera manager ini 

in the 

 

[general ] 

server_host, server_port, listening_port, listening_hostname

 

also whats you hostname that you had configured. 

avatar
Explorer

Hi this is my ini file:

[General]

# Hostname of the CM server.

server_host=xx.yyy.zz.aaa

 

# Port that the CM server is listening on.

server_port=7182

 

## It should not normally be necessary to modify these.

# Port that the CM agent should listen on.

# listening_port=9000

 

# IP Address that the CM agent should listen on.

# listening_ip=

 

# Hostname that the CM agent reports as its hostname. If unset, will be

# obtained in code through something like this:

#

#   python -c 'import socket; \

#              print socket.getfqdn(), \

#                    socket.gethostbyname(socket.getfqdn())'

#

# listening_hostname=

 

# An alternate hostname to report as the hostname for this host in CM.

# Useful when this agent is behind a load balancer or proxy and all

# inbound communication must connect through that proxy.

# reported_hostname=

 

# Port that supervisord should listen on.

# NB: This only takes effect if supervisord is restarted.

# supervisord_port=19001

 

# Log file.  The supervisord log file will be placed into

# the same directory.  Note that if the agent is being started via the

# init.d script, /var/log/cloudera-scm-agent/cloudera-scm-agent.out will

# also have a small amount of output (from before logging is initialized).

# log_file=/var/log/cloudera-scm-agent/cloudera-scm-agent.log

 

# Persistent state directory.  Directory to store CM agent state that

# persists across instances of the agent process and system reboots.

# Particularly, the agent's UUID is stored here.

# lib_dir=/var/lib/cloudera-scm-agent

 

# Parcel directory.  Unpacked parcels will be stored in this directory.

# Downloaded parcels will be stored in <parcel_dir>/../parcel-cache

# parcel_dir=/opt/cloudera/parcels

 

# Enable supervisord event monitoring.  Used in eager heartbeating, amongst

# other things.

# enable_supervisord_events=true

 

# Maximum time to wait (in seconds) for all metric collectors to finish

# collecting data.

max_collection_wait_seconds=10.0

 

# Maximum time to wait (in seconds) when connecting to a local role's

# webserver to fetch metrics.

metrics_url_timeout_seconds=30.0

 

# Maximum time to wait (in seconds) when connecting to a local TaskTracker

# to fetch task attempt data.

task_metrics_timeout_seconds=5.0

 

# The list of non-device (nodev) filesystem types which will be monitored.

monitored_nodev_filesystem_types=nfs,nfs4,tmpfs

 

# The list of filesystem types which are considered local for monitoring purposes.

# These filesystems are combined with the other local filesystem types found in

# /proc/filesystems

local_filesystem_whitelist=ext2,ext3,ext4,xfs

 

# The largest size impala profile log bundle that this agent will serve to the

# CM server. If the CM server requests more than this amount, the bundle will

# be limited to this size. All instances of this limit being hit are logged to

# the agent log.

impala_profile_bundle_max_bytes=1073741824

 

# The largest size stacks log bundle that this agent will serve to the CM

# server. If the CM server requests more than this amount, the bundle will be

# limited to this size. All instances of this limit being hit are logged to the

# agent log.

stacks_log_bundle_max_bytes=1073741824

 

# The size to which the uncompressed portion of a stacks log can grow before it

# is rotated. The log will then be compressed during rotation.

stacks_log_max_uncompressed_file_size_bytes=5242880

 

# The orphan process directory staleness threshold. If a diretory is more stale

# than this amount of seconds, CM agent will remove it.

orphan_process_dir_staleness_threshold=5184000

 

# The orphan process directory refresh interval. The CM agent will check the

# staleness of the orphan processes config directory every this amount of

# seconds.

orphan_process_dir_refresh_interval=3600

 

# A knob to control the agent logging level. The options are listed as follows:

# 1) DEBUG (set the agent logging level to 'logging.DEBUG')

# 2) INFO (set the agent logging level to 'logging.INFO')

scm_debug=INFO

 

# The DNS resolution collecion interval in seconds. A java base test program

# will be executed with at most this frequency to collect java DNS resolution

# metrics. The test program is only executed if the associated health test,

# Host DNS Resolution, is enabled.

dns_resolution_collection_interval_seconds=60

 

# The maximum time to wait (in seconds) for the java test program to collect

# java DNS resolution metrics.

dns_resolution_collection_timeout_seconds=30

 

# The directory location in which the agent-wide kerberos credential cache

# will be created.

# agent_wide_credential_cache_location=/var/run/cloudera-scm-agent

 

[Security]

# Use TLS and certificate validation when connecting to the CM server.

use_tls=1

 

# The maximum allowed depth of the certificate chain returned by the peer.

# The default value of 9 matches the default specified in openssl's

# SSL_CTX_set_verify.

max_cert_depth=9

 

# A file of CA certificates in PEM format. The file can contain several CA

# certificates identified by

#

# -----BEGIN CERTIFICATE-----

# ... (CA certificate in base64 encoding) ...

# -----END CERTIFICATE-----

#

# sequences. Before, between, and after the certificates text is allowed which

# can be used e.g. for descriptions of the certificates.

#

# The file is loaded once, the first time an HTTPS connection is attempted. A

# restart of the agent is required to pick up changes to the file.

#

# Note that if neither verify_cert_file or verify_cert_dir is set, certificate

# verification will not be performed.

# verify_cert_file=

 

# Directory containing CA certificates in PEM format. The files each contain one

# CA certificate. The files are looked up by the CA subject name hash value,

# which must hence be available. If more than one CA certificate with the same

# name hash value exist, the extension must be different (e.g. 9d66eef0.0,

# 9d66eef0.1 etc). The search is performed in the ordering of the extension

# number, regardless of other properties of the certificates. Use the c_rehash

# utility to create the necessary links.

#

# The certificates in the directory are only looked up when required, e.g. when

# building the certificate chain or when actually performing the verification

# of a peer certificate. The contents of the directory can thus be changed

# without an agent restart.

#

# When looking up CA certificates, the verify_cert_file is first searched, then

# those in the directory. Certificate matching is done based on the subject name,

# the key identifier (if present), and the serial number as taken from the

# certificate to be verified. If these data do not match, the next certificate

# will be tried. If a first certificate matching the parameters is found, the

# verification process will be performed; no other certificates for the same

# parameters will be searched in case of failure.

#

# Note that if neither verify_cert_file or verify_cert_dir is set, certificate

# verification will not be performed.

# verify_cert_dir=

 

# PEM file containing client private key.

# client_key_file=

 

# A command to run which returns the client private key password on stdout

# client_keypw_cmd=

 

# If client_keypw_cmd isn't specified, instead a text file containing

# the client private key password can be used.

# client_keypw_file=

 

# PEM file containing client certificate.

# client_cert_file=

 

## Location of Hadoop files.  These are the CDH locations when installed by

## packages.  Unused when CDH is installed by parcels.

[Hadoop]

#cdh_crunch_home=/usr/lib/crunch

#cdh_flume_home=/usr/lib/flume-ng

#cdh_hadoop_bin=/usr/bin/hadoop

#cdh_hadoop_home=/usr/lib/hadoop

#cdh_hbase_home=/usr/lib/hbase

#cdh_hbase_indexer_home=/usr/lib/hbase-solr

#cdh_hcat_home=/usr/lib/hive-hcatalog

#cdh_hdfs_home=/usr/lib/hadoop-hdfs

#cdh_hive_home=/usr/lib/hive

#cdh_httpfs_home=/usr/lib/hadoop-httpfs

#cdh_hue_home=/usr/share/hue

#cdh_hue_plugins_home=/usr/lib/hadoop

#cdh_impala_home=/usr/lib/impala

#cdh_llama_home=/usr/lib/llama

#cdh_mr1_home=/usr/lib/hadoop-0.20-mapreduce

#cdh_mr2_home=/usr/lib/hadoop-mapreduce

#cdh_oozie_home=/usr/lib/oozie

#cdh_parquet_home=/usr/lib/parquet

#cdh_pig_home=/usr/lib/pig

#cdh_solr_home=/usr/lib/solr

#cdh_spark_home=/usr/lib/spark

#cdh_sqoop_home=/usr/lib/sqoop

#cdh_sqoop2_home=/usr/lib/sqoop2

#cdh_yarn_home=/usr/lib/hadoop-yarn

#cdh_zookeeper_home=/usr/lib/zookeeper

#hive_default_xml=/etc/hive/conf.dist/hive-default.xml

#webhcat_default_xml=/etc/hive-webhcat/conf.dist/webhcat-default.xml

#jsvc_home=/usr/libexec/bigtop-utils

#tomcat_home=/usr/lib/bigtop-tomcat

#oracle_home=/usr/share/oracle/instantclient

 

## Location of Cloudera Management Services files.

[Cloudera]

#mgmt_home=/usr/share/cmf

 

## Location of JDBC Drivers.

[JDBC]

#cloudera_mysql_connector_jar=/usr/share/java/mysql-connector-java.jar

#cloudera_oracle_connector_jar=/usr/share/java/oracle-connector-java.jar

#By default, postgres jar is found dynamically in $MGMT_HOME/lib

#cloudera_postgresql_jdbc_jar=

 

 

 

and my hostname is amit.

and /etc/hosts file has:

xx.yyy.zz.aaa       amit

 

Kindly tell me where i am doing mistake.

avatar
Explorer

Hi,

As due to unknown reasons i'm unable to post my complete error logs in my message, i've attached a link to log. Please have a look on error log.

Thank You
Amit

avatar
Champion

i see that your host file has xxx yy something 

 

my etc/hosts file contains:
xx.yyy.zz.aaa     amit

and my hostname file contains:
amit

you should have some meaning full ip add like 192.168.138.2 

 

avatar
Explorer

the ip address is 10.239.39.115

 

Is there is anything where i need to do some changes to keep things going.

 

 

Thanks,
Amit