Created 08-24-2017 04:44 AM
Error Message: Installation failed. Failed to receive heartbeat from agent. • Ensure that the host's hostname is configured properly. • Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules). • Ensure that ports 9000 and 9001 are not in use on the host being added. • Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details). • If Use TLS Encryption for Agentsis enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.inihas use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here. Log file: /tmp/scm_prepare_node.bK5vQRgw using SSH_CLIENT to get the SCM hostname: 10.x.y.z 38830 22 opening logging file descriptor Starting installation script... Acquiring installation lock... BEGIN flock 4 END (0) Detecting root privileges... effective UID is 0 Detecting distribution... BEGIN grep 'Ubuntu' /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS" END (0) BEGIN grep DISTRIB_CODENAME /etc/lsb-release END (0) DISTRIB_CODENAME=xenial BEGIN echo DISTRIB_CODENAME=xenial | cut -d = -f 2 xenial END (0) Detecting Cloudera Manager Server... BEGIN host -t PTR 10.x.y.z Host z.y.x.10.in-addr.arpa. not found: 3(NXDOMAIN) END (1) BEGIN which python /usr/bin/python END (0) BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' 10.x.y.z 7182 END (0) BEGIN which wget /usr/bin/wget END (0) BEGIN wget -qO- -T 1 -t 1 http://169.254.169.254/latest/meta-data/public-hostname && /bin/echo END (4) Installing package repositories... validating format of repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list installing repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list repository file /tmp/scm_prepare_node.bK5vQRgw/repos/ubuntu_xenial/cloudera-manager.list installed BEGIN apt-key add /tmp/scm_prepare_node.bK5vQRgw/archive.key OK END (0) installing priority file /tmp/scm_prepare_node.bK5vQRgw/priorities/ubuntu_xenial/cloudera.pref priority file /tmp/scm_prepare_node.bK5vQRgw/priorities/ubuntu_xenial/cloudera.pref in-stalled Refreshing package metadata... BEGIN apt-get update Hit:1 http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5 InRelease Hit:2 https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 InRe-lease Hit:3 http://repo.mysql.com/apt/ubuntu xenial InRelease Hit:4 https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5.12.0 InRe-lease Hit:5 http://in.archive.ubuntu.com/ubuntu xenial InRelease Get:6 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] Get:7 http://in.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB] Get:8 http://security.ubuntu.com/ubuntu xenial-security/main amd64 DEP-11 Metadata [60.2 kB] Get:9 http://security.ubuntu.com/ubuntu xenial-security/main DEP-11 64x64 Icons [52.0 kB] Get:10 http://in.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB] Get:11 http://in.archive.ubuntu.com/ubuntu xenial-updates/main amd64 DEP-11 Metada-ta [305 kB] Get:12 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 DEP-11 Metadata [48.8 kB] Get:13 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [64.2 kB] Get:14 http://in.archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons [206 kB] Get:15 http://in.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [171 kB] Get:16 http://in.archive.ubuntu.com/ubuntu xenial-updates/universe DEP-11 64x64 Icons [224 kB] Get:17 http://in.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 DEP-11 Metadata [5,892 B] Get:18 http://in.archive.ubuntu.com/ubuntu xenial-backports/main amd64 DEP-11 Metadata [3,324 B] Get:19 http://in.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 DEP-11 Metadata [5,136 B] Fetched 1,451 kB in 6s (241 kB/s) Reading package lists... W: http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/dists/precise-cdh5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.12.0/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) END (0) BEGIN apt-get update Hit:1 http://repo.mysql.com/apt/ubuntu xenial InRelease Hit:2 http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5 InRelease Hit:3 https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh precise-cdh5 InRe-lease Hit:4 https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm xenial-cm5.12.0 InRe-lease Hit:5 http://security.ubuntu.com/ubuntu xenial-security InRelease Hit:6 http://in.archive.ubuntu.com/ubuntu xenial InRelease Hit:7 http://in.archive.ubuntu.com/ubuntu xenial-updates InRelease Hit:8 http://in.archive.ubuntu.com/ubuntu xenial-backports InRelease Reading package lists... W: http://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/dists/precise-cdh5/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) W: https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.12.0/InRelease: Signature by key F36A89E33CC1BD0F71079007327574EE02A818DD uses weak digest algorithm (SHA1) END (0) Installing cloudera-manager-agent package... BEGIN dpkg -l cloudera-manager-agent | grep -E '^ii[[:space:]]*cloudera-manager-agent[[:space:]]*' ii cloudera-manager-agent 5.12.0-1.cm5120.p0.120~xenial-cm5 amd64 The Cloudera Manager Agent END (0) BEGIN echo oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-agent oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons END (0) BEGIN apt-cache show cloudera-manager-agent Package: cloudera-manager-agent Source: enterprise Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 Architecture: amd64 Maintainer: Cloudera Inc. Installed-Size: 44039 Depends: libc6 (>= 2.17), libcomerr2 (>= 1.01), libgcc1 (>= 1:3.0), libgssapi-krb5-2 (>= 1.10+dfsg~), libkrb5-3 (>= 1.6.dfsg.2), libssl1.0.0 (>= 1.0.2~beta3), libstdc++6 (>= 5.2), zlib1g (>= 1:1.2.0), lsb-base, psmisc, bash, libsasl2-modules, libsasl2-modules-gssapi-mit, libxslt1.1, libsqlite3-0, openssl, libssl-dev, libfuse2, fuse-utils | fuse, rpcbind, cloudera-manager-daemons (= 5.12.0-1.cm5120.p0.120~xenial-cm5), perl, python-psycopg2, py-thon-mysqldb, apache2, iproute2 Homepage: http://www.cloudera.com Priority: extra Section: misc Filename: pool/contrib/e/enterprise/cloudera-manager-agent_5.12.0-1.cm5120.p0.120~xenial-cm5_amd64.deb Size: 11186376 SHA256: 69618949fb10d631ac301f21f62ecab0a3631eec456b6de43ef9c6a17c107498 SHA1: 6b137a4328780d56f964403ff8c56fe7577935a0 MD5sum: f7c9fc2b393b6d61563ea34ca15b6e61 Description: The Cloudera Manager Agent The Agent is deployed to machines running services managed by Cloudera Manager. Description-md5: 482a592ee859d31443f58aa3e48fe303 END (0) Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 BEGIN apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y in-stall cloudera-manager-agent Reading package lists... Building dependency tree... Reading state information... cloudera-manager-agent is already the newest version (5.12.0-1.cm5120.p0.120~xenial-cm5). 0 upgraded, 0 newly installed, 0 to remove and 347 not upgraded. END (0) remote package cloudera-manager-agent installed Installing cloudera-manager-daemons package... BEGIN dpkg -l cloudera-manager-daemons | grep -E '^ii[[:space:]]*cloudera-manager-daemons[[:space:]]*' ii cloudera-manager-daemons 5.12.0-1.cm5120.p0.120~xenial-cm5 all Provides daemons for monitoring Hadoop and related tools. END (0) BEGIN echo oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-daemons oracle-j2sdk1.6 oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons END (0) BEGIN apt-cache show cloudera-manager-daemons Package: cloudera-manager-daemons Source: enterprise Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 Architecture: all Maintainer: Cloudera Inc. Installed-Size: 869404 Depends: perl Homepage: http://www.cloudera.com Priority: extra Section: misc Filename: pool/contrib/e/enterprise/cloudera-manager-daemons_5.12.0-1.cm5120.p0.120~xenial-cm5_all.deb Size: 737883268 SHA256: cf168ac13387bef54c61034e7cfef49539b1616cfb876f91db5dfe458d237780 SHA1: a18ed00de131d0c358411104fb3d886e0c211101 MD5sum: 47813b5493281f58b36ca33868229394 Description: Provides daemons for monitoring Hadoop and related tools. Provides daemons for monitoring Hadoop and related tools. Description-md5: 35bb5bc8fc2f296f791c3bda9e345340 END (0) Version: 5.12.0-1.cm5120.p0.120~xenial-cm5 BEGIN apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y in-stall cloudera-manager-daemons Reading package lists... Building dependency tree... Reading state information... cloudera-manager-daemons is already the newest version (5.12.0-1.cm5120.p0.120~xenial-cm5). 0 upgraded, 0 newly installed, 0 to remove and 347 not upgraded. END (0) remote package cloudera-manager-daemons installed Installing Unlimited Strength Encryption policy files. Installation not requested. Step will be skipped. Configuring Cloudera Manager Agent... BEGIN grep server_host=10.x.y.z /etc/cloudera-scm-agent/config.ini server_host=10.x.y.z END (0) scm agent is already configured BEGIN sed -e 's/#\(USER=.*\)/\1/' -i /etc/default/cloudera-scm-agent END (0) Starting Cloudera Manager Agent... BEGIN /usr/sbin/service cloudera-scm-agent status * cloudera-scm-agent.service - LSB: Cloudera SCM Agent Loaded: loaded (/etc/init.d/cloudera-scm-agent; bad; vendor preset: enabled) Active: active (exited) since Thu 2017-08-24 15:47:19 IST; 58min ago Docs: man:systemd-sysv-generator(8) Process: 9473 ExecStop=/etc/init.d/cloudera-scm-agent stop (code=exited, sta-tus=0/SUCCESS) Process: 9496 ExecStart=/etc/init.d/cloudera-scm-agent start (code=exited, sta-tus=0/SUCCESS) Aug 24 15:47:17 PC355406 systemd[1]: Starting LSB: Cloudera SCM Agent... Aug 24 15:47:17 PC355406 su[9516]: Successful su for cloudera-scm by root Aug 24 15:47:17 PC355406 su[9516]: + ??? root:cloudera-scm Aug 24 15:47:17 PC355406 su[9516]: pam_unix(su:session): session opened for user cloudera-scm by (uid=0) Aug 24 15:47:19 PC355406 cloudera-scm-agent[9496]: Starting cloudera-scm-agent: * cloudera-scm-agent started Aug 24 15:47:19 PC355406 systemd[1]: Started LSB: Cloudera SCM Agent. END (0) BEGIN /usr/sbin/service cloudera-scm-agent restart END (0) agent logs: BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/' >>[24/Aug/2017 12:55:20 +0000] 1442 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent >>[24/Aug/2017 12:55:20 +0000] 1442 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:03:49 +0000] 1426 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 13:06:32 +0000] 4147 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:38:40 +0000] 6308 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:40:25 +0000] 7651 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:43:14 +0000] 8111 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 15:47:19 +0000] 9518 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:260: SyntaxWarning: name 'REDACTOR' is assigned to be-fore global declaration >> global REDACTOR >>/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/redaction.py:261: SyntaxWarning: name 'NOP_REDACTOR' is assigned to before global declaration >> global NOP_REDACTOR >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO SCM Agent Version: 5.12.0 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755 >>[24/Aug/2017 16:46:11 +0000] 11744 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent END (0) BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/' >>[24/Aug/2017 15:47:19 +0000] 9533 MainThread agent INFO Because agent not run-ning as root, all processes will run with current user. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/cgroups >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem blkio at /sys/fs/cgroup/blkio >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem memory at /sys/fs/cgroup/memory >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpu at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpuacct at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups WARNING Could not read capabilities from /proc/cgroups. error: sudo: no tty present and no askpass program speci-fied >> >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO No cgroups capabilities found >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Setting up supervisord event monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using join timeout of 0.100000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using tolerance of 60.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread kt_renewer INFO Agent wide creden-tial cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_123 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using met-rics_url_timeout_seconds of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread ntp_monitor INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread dns_names INFO Using timeout of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Created DNS monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread stacks_collection_manager INFO Us-ing max_uncompressed_file_size_bytes: 5242880 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Importing metric sche-ma from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/share/maven', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUS-TEE_SERVER_HOME': '/usr/lib/keyt>>[24/Aug/2017 15:47:19 +0000] 9533 MainThread agent INFO Because agent not running as root, all processes will run with current user. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/cgroups >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem blkio at /sys/fs/cgroup/blkio >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem memory at /sys/fs/cgroup/memory >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpu at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups INFO Found existing subsys-tem cpuacct at /sys/fs/cgroup/cpu,cpuacct >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread cgroups WARNING Could not read capabilities from /proc/cgroups. error: sudo: no tty present and no askpass program speci-fied >> >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO No cgroups capabilities found >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Setting up supervisord event monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using join timeout of 0.100000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Using tolerance of 60.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs'] >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread kt_renewer INFO Agent wide creden-tial cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_123 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using met-rics_url_timeout_seconds of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Using max_collection_wait_seconds of 10.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread ntp_monitor INFO Using timeout of 2.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread dns_names INFO Using timeout of 30.000000 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Created DNS monitor. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread stacks_collection_manager INFO Us-ing max_uncompressed_file_size_bytes: 5242880 >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread __init__ INFO Importing metric sche-ma from file /usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/share/maven', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUS-TEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'SUDO': 'sudo', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO To override these varia-bles, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/process >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/flood >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor/include >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent ERROR Failed to connect to previous supervisor. >>Traceback (most recent call last): >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor >> self.configure_supervisor_clients() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients >> supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervi-sord.conf")]) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize >> Options.realize(self, *arg, **kw) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize >> self.process_config() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config >> self.process_config_file(do_usage) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file >> self.usage(str(msg)) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage >> self.exit(2) >>SystemExit: 2 >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 daemonize WARNING Stopping dae-mon. >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 agent INFO Stopping agent... rustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'SUDO': 'sudo', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'} >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO To override these varia-bles, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels. >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/process >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/flood >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent INFO Re-using pre-existing di-rectory: /run/cloudera-scm-agent/supervisor/include >>[24/Aug/2017 15:47:21 +0000] 9533 MainThread agent ERROR Failed to connect to previous supervisor. >>Traceback (most recent call last): >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor >> self.configure_supervisor_clients() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients >> supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervi-sord.conf")]) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize >> Options.realize(self, *arg, **kw) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize >> self.process_config() >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config >> self.process_config_file(do_usage) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file >> self.usage(str(msg)) >> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage >> self.exit(2) >>SystemExit: 2 >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 daemonize WARNING Stopping dae-mon. >>[24/Aug/2017 15:47:21 +0000] 9533 Dummy-1 agent INFO Stopping agent... END (0) end of agent logs. scm agent restarted Installation script completed successfully. all done closing logging file descriptor My /etc/hosts file is: 127.0.0.1 localhost #127.0.0.1 PC355406 10.x.y.z PC355406 # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters Kindly Provide me a solution.
Created 08-26-2017 11:36 PM
your /etc/hosts file should be like this
192.168.121.11 cluster1
also change your hostname to cluster1
/etc/hostname
finally if its a test enviroment either disable the firewall or change the rules accordingly .
this should fix you the problem.
are you configuring single or multi node cluster ?
Created on 08-27-2017 10:39 PM - edited 08-27-2017 10:49 PM
I am configuring for single node only. I am new to the ubuntu system so can u please tell me what is the difference between single node and multi node.
Also in /etc/hosts i have to remove all the configuration and just put:
x.y.z.a cluster1
127.0.0.1 is not required in /etc/hosts file ??????
Currently my /etc/hosts file is like this:
My /etc/hosts file is:
127.0.0.1 localhost
#127.0.0.1 PC355406
10.x.y.z PC355406
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
Kindly help me in that.
Created 08-28-2017 03:15 AM
single node cluster - you will have all the service under one host mostly for testing purpose .
multi node cluster - you will have many host each one will have their own dameon runining for example
node1 will have master service like - namenode , YARN
node 2 will have impala daemons
node 3 will have hive server2
node 4 - can acts edge nodes
to make it simple for a single node cluster configure like this
/etc/hosts
any ip address like (192.168.121.14) server1
/etc/hostname will have
server1
exit the terminal or reboot the system - make sure you have your hostname configured properly by typing
echo $hostname.
Created on 08-28-2017 03:34 AM - edited 08-28-2017 04:06 AM
hi,
i have tried the solution you suggested but still i got sam error.
i have disabled iptables,
i have also put use_tls=1 in the file /etc/cloudera-scm-agent/config.inihas
what else i can do to solve this problem and how to do that as i am new to this environment??
Thanks in advance for helping me..
Created 08-28-2017 04:28 AM
tell me what you have under your cloudera manager ini
in the
[general ]
server_host, server_port, listening_port, listening_hostname
also whats you hostname that you had configured.
Created 08-28-2017 06:11 AM
Hi this is my ini file:
[General]
# Hostname of the CM server.
server_host=xx.yyy.zz.aaa
# Port that the CM server is listening on.
server_port=7182
## It should not normally be necessary to modify these.
# Port that the CM agent should listen on.
# listening_port=9000
# IP Address that the CM agent should listen on.
# listening_ip=
# Hostname that the CM agent reports as its hostname. If unset, will be
# obtained in code through something like this:
#
# python -c 'import socket; \
# print socket.getfqdn(), \
# socket.gethostbyname(socket.getfqdn())'
#
# listening_hostname=
# An alternate hostname to report as the hostname for this host in CM.
# Useful when this agent is behind a load balancer or proxy and all
# inbound communication must connect through that proxy.
# reported_hostname=
# Port that supervisord should listen on.
# NB: This only takes effect if supervisord is restarted.
# supervisord_port=19001
# Log file. The supervisord log file will be placed into
# the same directory. Note that if the agent is being started via the
# init.d script, /var/log/cloudera-scm-agent/cloudera-scm-agent.out will
# also have a small amount of output (from before logging is initialized).
# log_file=/var/log/cloudera-scm-agent/cloudera-scm-agent.log
# Persistent state directory. Directory to store CM agent state that
# persists across instances of the agent process and system reboots.
# Particularly, the agent's UUID is stored here.
# lib_dir=/var/lib/cloudera-scm-agent
# Parcel directory. Unpacked parcels will be stored in this directory.
# Downloaded parcels will be stored in <parcel_dir>/../parcel-cache
# parcel_dir=/opt/cloudera/parcels
# Enable supervisord event monitoring. Used in eager heartbeating, amongst
# other things.
# enable_supervisord_events=true
# Maximum time to wait (in seconds) for all metric collectors to finish
# collecting data.
max_collection_wait_seconds=10.0
# Maximum time to wait (in seconds) when connecting to a local role's
# webserver to fetch metrics.
metrics_url_timeout_seconds=30.0
# Maximum time to wait (in seconds) when connecting to a local TaskTracker
# to fetch task attempt data.
task_metrics_timeout_seconds=5.0
# The list of non-device (nodev) filesystem types which will be monitored.
monitored_nodev_filesystem_types=nfs,nfs4,tmpfs
# The list of filesystem types which are considered local for monitoring purposes.
# These filesystems are combined with the other local filesystem types found in
# /proc/filesystems
local_filesystem_whitelist=ext2,ext3,ext4,xfs
# The largest size impala profile log bundle that this agent will serve to the
# CM server. If the CM server requests more than this amount, the bundle will
# be limited to this size. All instances of this limit being hit are logged to
# the agent log.
impala_profile_bundle_max_bytes=1073741824
# The largest size stacks log bundle that this agent will serve to the CM
# server. If the CM server requests more than this amount, the bundle will be
# limited to this size. All instances of this limit being hit are logged to the
# agent log.
stacks_log_bundle_max_bytes=1073741824
# The size to which the uncompressed portion of a stacks log can grow before it
# is rotated. The log will then be compressed during rotation.
stacks_log_max_uncompressed_file_size_bytes=5242880
# The orphan process directory staleness threshold. If a diretory is more stale
# than this amount of seconds, CM agent will remove it.
orphan_process_dir_staleness_threshold=5184000
# The orphan process directory refresh interval. The CM agent will check the
# staleness of the orphan processes config directory every this amount of
# seconds.
orphan_process_dir_refresh_interval=3600
# A knob to control the agent logging level. The options are listed as follows:
# 1) DEBUG (set the agent logging level to 'logging.DEBUG')
# 2) INFO (set the agent logging level to 'logging.INFO')
scm_debug=INFO
# The DNS resolution collecion interval in seconds. A java base test program
# will be executed with at most this frequency to collect java DNS resolution
# metrics. The test program is only executed if the associated health test,
# Host DNS Resolution, is enabled.
dns_resolution_collection_interval_seconds=60
# The maximum time to wait (in seconds) for the java test program to collect
# java DNS resolution metrics.
dns_resolution_collection_timeout_seconds=30
# The directory location in which the agent-wide kerberos credential cache
# will be created.
# agent_wide_credential_cache_location=/var/run/cloudera-scm-agent
[Security]
# Use TLS and certificate validation when connecting to the CM server.
use_tls=1
# The maximum allowed depth of the certificate chain returned by the peer.
# The default value of 9 matches the default specified in openssl's
# SSL_CTX_set_verify.
max_cert_depth=9
# A file of CA certificates in PEM format. The file can contain several CA
# certificates identified by
#
# -----BEGIN CERTIFICATE-----
# ... (CA certificate in base64 encoding) ...
# -----END CERTIFICATE-----
#
# sequences. Before, between, and after the certificates text is allowed which
# can be used e.g. for descriptions of the certificates.
#
# The file is loaded once, the first time an HTTPS connection is attempted. A
# restart of the agent is required to pick up changes to the file.
#
# Note that if neither verify_cert_file or verify_cert_dir is set, certificate
# verification will not be performed.
# verify_cert_file=
# Directory containing CA certificates in PEM format. The files each contain one
# CA certificate. The files are looked up by the CA subject name hash value,
# which must hence be available. If more than one CA certificate with the same
# name hash value exist, the extension must be different (e.g. 9d66eef0.0,
# 9d66eef0.1 etc). The search is performed in the ordering of the extension
# number, regardless of other properties of the certificates. Use the c_rehash
# utility to create the necessary links.
#
# The certificates in the directory are only looked up when required, e.g. when
# building the certificate chain or when actually performing the verification
# of a peer certificate. The contents of the directory can thus be changed
# without an agent restart.
#
# When looking up CA certificates, the verify_cert_file is first searched, then
# those in the directory. Certificate matching is done based on the subject name,
# the key identifier (if present), and the serial number as taken from the
# certificate to be verified. If these data do not match, the next certificate
# will be tried. If a first certificate matching the parameters is found, the
# verification process will be performed; no other certificates for the same
# parameters will be searched in case of failure.
#
# Note that if neither verify_cert_file or verify_cert_dir is set, certificate
# verification will not be performed.
# verify_cert_dir=
# PEM file containing client private key.
# client_key_file=
# A command to run which returns the client private key password on stdout
# client_keypw_cmd=
# If client_keypw_cmd isn't specified, instead a text file containing
# the client private key password can be used.
# client_keypw_file=
# PEM file containing client certificate.
# client_cert_file=
## Location of Hadoop files. These are the CDH locations when installed by
## packages. Unused when CDH is installed by parcels.
[Hadoop]
#cdh_crunch_home=/usr/lib/crunch
#cdh_flume_home=/usr/lib/flume-ng
#cdh_hadoop_bin=/usr/bin/hadoop
#cdh_hadoop_home=/usr/lib/hadoop
#cdh_hbase_home=/usr/lib/hbase
#cdh_hbase_indexer_home=/usr/lib/hbase-solr
#cdh_hcat_home=/usr/lib/hive-hcatalog
#cdh_hdfs_home=/usr/lib/hadoop-hdfs
#cdh_hive_home=/usr/lib/hive
#cdh_httpfs_home=/usr/lib/hadoop-httpfs
#cdh_hue_home=/usr/share/hue
#cdh_hue_plugins_home=/usr/lib/hadoop
#cdh_impala_home=/usr/lib/impala
#cdh_llama_home=/usr/lib/llama
#cdh_mr1_home=/usr/lib/hadoop-0.20-mapreduce
#cdh_mr2_home=/usr/lib/hadoop-mapreduce
#cdh_oozie_home=/usr/lib/oozie
#cdh_parquet_home=/usr/lib/parquet
#cdh_pig_home=/usr/lib/pig
#cdh_solr_home=/usr/lib/solr
#cdh_spark_home=/usr/lib/spark
#cdh_sqoop_home=/usr/lib/sqoop
#cdh_sqoop2_home=/usr/lib/sqoop2
#cdh_yarn_home=/usr/lib/hadoop-yarn
#cdh_zookeeper_home=/usr/lib/zookeeper
#hive_default_xml=/etc/hive/conf.dist/hive-default.xml
#webhcat_default_xml=/etc/hive-webhcat/conf.dist/webhcat-default.xml
#jsvc_home=/usr/libexec/bigtop-utils
#tomcat_home=/usr/lib/bigtop-tomcat
#oracle_home=/usr/share/oracle/instantclient
## Location of Cloudera Management Services files.
[Cloudera]
#mgmt_home=/usr/share/cmf
## Location of JDBC Drivers.
[JDBC]
#cloudera_mysql_connector_jar=/usr/share/java/mysql-connector-java.jar
#cloudera_oracle_connector_jar=/usr/share/java/oracle-connector-java.jar
#By default, postgres jar is found dynamically in $MGMT_HOME/lib
#cloudera_postgresql_jdbc_jar=
and my hostname is amit.
and /etc/hosts file has:
xx.yyy.zz.aaa amit
Kindly tell me where i am doing mistake.
Created 08-28-2017 11:31 PM
Hi,
As due to unknown reasons i'm unable to post my complete error logs in my message, i've attached a link to log. Please have a look on error log.
Thank You
Amit
Created 08-29-2017 12:07 AM
i see that your host file has xxx yy something
my etc/hosts file contains: xx.yyy.zz.aaa amit and my hostname file contains: amit
you should have some meaning full ip add like 192.168.138.2
Created 08-29-2017 01:55 AM
the ip address is 10.239.39.115
Is there is anything where i need to do some changes to keep things going.
Thanks,
Amit