question Re: CDH 5.12.0 clouder manager agent can not start. in Support Questions

CDH 5.12.0 clouder manager agent can not start.

leiyz — Fri, 16 Sep 2022 12:04:13 GMT

this is log file of cloudera-scm-agent.log:

....

INFO Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
[08/Aug/2017 20:35:43 +0000] 88588 MainThread agent ERROR Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor
self.configure_supervisor_clients()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients
supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")])
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize
Options.realize(self, *arg, **kw)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize
self.process_config()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config
self.process_config_file(do_usage)
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Created /run/cloudera-scm-agent/process
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/process to 0751
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Created /run/cloudera-scm-agent/supervisor
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Created /run/cloudera-scm-agent/flood
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/flood to 0751
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Created /run/cloudera-scm-agent/supervisor/include
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
[10/Aug/2017 14:02:42 +0000] 2675 MainThread agent ERROR Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor
self.configure_supervisor_clients()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients
supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")])
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize
Options.realize(self, *arg, **kw)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize
self.process_config()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config
self.process_config_file(do_usage)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file
self.usage(str(msg))
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage
self.exit(2)
SystemExit: 2
[10/Aug/2017 14:02:42 +0000] 2675 Dummy-1 daemonize WARNING Stopping daemon.
[10/Aug/2017 14:02:42 +0000] 2675 Dummy-1 agent INFO Stopping agent...
[10/Aug/2017 14:02:42 +0000] 2675 Dummy-1 agent INFO No extant cgroups; unmounting any cgroup roots

============================================

this is log file of cloudera-scm-agent.out：

[09/Aug/2017 17:49:35 +0000] 2179 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
[10/Aug/2017 14:02:39 +0000] 1038 MainThread agent INFO SCM Agent Version: 5.12.0
[10/Aug/2017 14:02:39 +0000] 1038 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
[10/Aug/2017 14:02:39 +0000] 1038 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent

============

is anyone can help me ?ths...

Re: CDH 5.12.0 clouder manager agent can not start.

leiyz — Thu, 10 Aug 2017 06:50:08 GMT

some configuration is like this:

[root@bigdata-master cloudera-scm-agent]# cat /etc/hosts
#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
192.168.1.127 bigdata-master.lcloud.com bigdata-master
192.168.1.128 bigdata-slave1.lcloud.com bigdata-slave1
192.168.1.129 bigdata-slave2.lcloud.com bigdata-slave2

=========

[root@bigdata-master cloudera-scm-agent]# cat /etc/hostname
bigdata-master

===============

[root@bigdata-master cloudera-scm-agent]# cat /etc/sysconfig/network
INETWORKING=yes
HOSTNAME=bigdata-master
NETWORKING_IPV6=no# Created by anaconda

also firewall is disable,and i using centos7.3

Re: CDH 5.12.0 clouder manager agent can not start.

michalis — Thu, 10 Aug 2017 10:06:29 GMT

hi leiyz,

can you share your mount output and also what does the following command show?

for mnt in $(mount|cut -d ' ' -f 3); do stat $mnt 1>/dev/null 2>&1; rc=$?; if [ $rc -ne 0 ]; then echo "error accessing $mnt"; fi; done

regards,

michalis

Re: CDH 5.12.0 clouder manager agent can not start.

leiyz — Fri, 11 Aug 2017 01:12:02 GMT

hi Michalis,when i run the command you give to me .i got this

[root@bigdata-master ~]# for mnt in $(mount|cut -d ' ' -f 3); do stat $mnt 1>/dev/null 2>&1; rc=$?; if [ $rc -ne 0 ]; then echo "error accessing $mnt"; fi; done
error accessing /run/user/42/gvfs

i run mount and df -h commands.the output are below:

[root@bigdata-master ~]# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,size=4064920k,nr_inodes=1016230,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/sda3 on / type xfs (rw,relatime,attr2,inode64,noquota)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=35,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
/dev/sda1 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
tmpfs on /run/user/42 type tmpfs (rw,nosuid,nodev,relatime,size=815920k,mode=700,uid=42,gid=42)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
gvfsd-fuse on /run/user/42/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=42,group_id=42)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=815920k,mode=700)

[root@bigdata-master ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 92G 18G 75G 20% /
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 84K 3.9G 1% /dev/shm
tmpfs 3.9G 9.0M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/sda1 297M 152M 146M 51% /boot
tmpfs 797M 16K 797M 1% /run/user/42
tmpfs 797M 0 797M 0% /run/user/0

any problem is here?

thks for help.

Re: CDH 5.12.0 clouder manager agent can not start.

michalis — Fri, 11 Aug 2017 07:23:18 GMT

Hi leiyz,

You will need to umount /run/user/42/gvfs or /run/user/42 then you should be able to start the Cloudera Agent. You are indeed running into the same issue as josholsan in this case [0]

[0] http://community.cloudera.com/t5/Cloudera-Manager-Installation/Cloudera-Manager-Agent-cannot-connect-with-supervisor-Failed-to/td-p/58522

Re: CDH 5.12.0 clouder manager agent can not start.

leiyz — Fri, 11 Aug 2017 07:52:20 GMT

ok,thks.i will try to run umount /run/user/42/gvfs,but i still do not know why i should do this.

Re: CDH 5.12.0 clouder manager agent can not start.

leiyz — Fri, 11 Aug 2017 08:42:55 GMT

hi,Michalis.i have try the method you gave to me,and the agent start success.but i do not understand why during config on the admin web console,the cloudera manager still intall agent again.the log likes this.

Dependencies Resolved

================================================================================
Package Arch Version Repository Size
================================================================================
Updating:
cloudera-manager-agent
x86_64 5.12.0-1.cm5120.p0.120.el7 cloudera-manager 9.3 M
Updating for dependencies:
cloudera-manager-daemons
x86_64 5.12.0-1.cm5120.p0.120.el7 cloudera-manager 669 M

Transaction Summary
================================================================================
Upgrade 1 Package (+1 Dependent package)

Total download size: 678 M
Downloading packages:
No Presto metadata available for cloudera-manager

i have 3 node in my cluster,one of them is install success and one is failed for the reason below。

BEGIN /sbin/service cloudera-scm-agent status
Redirecting to /bin/systemctl status cloudera-scm-agent.service
Unit cloudera-scm-agent.service could not be found.
END (4)
BEGIN /sbin/service cloudera-scm-agent start
Redirecting to /bin/systemctl start cloudera-scm-agent.service
Failed to start cloudera-scm-agent.service: Unit not found.
END (5)
agent logs:
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/'
>>[08/Aug/2017 20:24:39 +0000] 65406 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 20:24:39 +0000] 65406 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 20:24:39 +0000] 65406 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 20:44:31 +0000] 66642 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 20:44:31 +0000] 66642 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 20:44:31 +0000] 66642 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 22:48:18 +0000] 69098 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 22:48:18 +0000] 69098 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 22:48:18 +0000] 69098 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 23:31:57 +0000] 70576 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 23:31:57 +0000] 70576 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 23:31:57 +0000] 70576 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 23:40:14 +0000] 2100 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 23:40:14 +0000] 2100 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 23:40:14 +0000] 2100 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[09/Aug/2017 00:54:00 +0000] 1043 MainThread agent INFO SCM Agent Version: 5.12.0
>>[09/Aug/2017 00:54:00 +0000] 1043 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[09/Aug/2017 00:54:00 +0000] 1043 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[09/Aug/2017 17:49:26 +0000] 2059 MainThread agent INFO SCM Agent Version: 5.12.0
>>[09/Aug/2017 17:49:26 +0000] 2059 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[09/Aug/2017 17:49:26 +0000] 2059 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[10/Aug/2017 17:16:18 +0000] 965 MainThread agent INFO SCM Agent Version: 5.12.0
>>[10/Aug/2017 17:16:18 +0000] 965 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[10/Aug/2017 17:16:18 +0000] 965 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 08:53:59 +0000] 2015 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 08:53:59 +0000] 2015 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 08:53:59 +0000] 2015 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 10:43:15 +0000] 4573 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 10:43:15 +0000] 4573 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 10:43:15 +0000] 4573 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 10:49:30 +0000] 5502 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 10:49:30 +0000] 5502 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 10:49:30 +0000] 5502 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 11:04:38 +0000] 6464 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 11:04:38 +0000] 6464 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 11:04:38 +0000] 6464 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 11:30:15 +0000] 6989 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 11:30:15 +0000] 6989 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 11:30:15 +0000] 6989 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 11:32:17 +0000] 7896 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 11:32:17 +0000] 7896 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 11:32:17 +0000] 7896 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 14:03:22 +0000] 10959 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 14:03:22 +0000] 10959 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 14:03:22 +0000] 10959 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 15:15:47 +0000] 1997 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 15:15:48 +0000] 1997 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 15:15:48 +0000] 1997 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 20:24:39 +0000] 65406 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 20:24:39 +0000] 65406 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 20:24:39 +0000] 65406 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 20:44:31 +0000] 66642 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 20:44:31 +0000] 66642 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 20:44:31 +0000] 66642 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 22:48:18 +0000] 69098 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 22:48:18 +0000] 69098 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 22:48:18 +0000] 69098 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 23:31:57 +0000] 70576 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 23:31:57 +0000] 70576 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 23:31:57 +0000] 70576 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[08/Aug/2017 23:40:14 +0000] 2100 MainThread agent INFO SCM Agent Version: 5.12.0
>>[08/Aug/2017 23:40:14 +0000] 2100 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[08/Aug/2017 23:40:14 +0000] 2100 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[09/Aug/2017 00:54:00 +0000] 1043 MainThread agent INFO SCM Agent Version: 5.12.0
>>[09/Aug/2017 00:54:00 +0000] 1043 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[09/Aug/2017 00:54:00 +0000] 1043 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[09/Aug/2017 17:49:26 +0000] 2059 MainThread agent INFO SCM Agent Version: 5.12.0
>>[09/Aug/2017 17:49:26 +0000] 2059 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[09/Aug/2017 17:49:26 +0000] 2059 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[10/Aug/2017 17:16:18 +0000] 965 MainThread agent INFO SCM Agent Version: 5.12.0
>>[10/Aug/2017 17:16:18 +0000] 965 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[10/Aug/2017 17:16:18 +0000] 965 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 08:53:59 +0000] 2015 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 08:53:59 +0000] 2015 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 08:53:59 +0000] 2015 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 10:43:15 +0000] 4573 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 10:43:15 +0000] 4573 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 10:43:15 +0000] 4573 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 10:49:30 +0000] 5502 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 10:49:30 +0000] 5502 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 10:49:30 +0000] 5502 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 11:04:38 +0000] 6464 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 11:04:38 +0000] 6464 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 11:04:38 +0000] 6464 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 11:30:15 +0000] 6989 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 11:30:15 +0000] 6989 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 11:30:15 +0000] 6989 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 11:32:17 +0000] 7896 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 11:32:17 +0000] 7896 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 11:32:17 +0000] 7896 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 14:03:22 +0000] 10959 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 14:03:22 +0000] 10959 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 14:03:22 +0000] 10959 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[11/Aug/2017 15:15:47 +0000] 1997 MainThread agent INFO SCM Agent Version: 5.12.0
>>[11/Aug/2017 15:15:48 +0000] 1997 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[11/Aug/2017 15:15:48 +0000] 1997 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
END (0)
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/'
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread cgroups INFO Found cgroups subsystem: cpuacct
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread cgroups INFO Found cgroups subsystem: memory
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread cgroups INFO Found cgroups subsystem: blkio
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread agent INFO Found cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes': 8796093022207, 'default_memory_soft_limit_in_bytes': 8796093022207, 'writable_cgroup_dot_procs': False, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True, 'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct': True, 'has_blkio': True}
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread agent INFO Setting up supervisord event monitor.
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread filesystem_map INFO Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread filesystem_map INFO Using timeout of 2.000000
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread filesystem_map INFO Using join timeout of 0.100000
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread filesystem_map INFO Using tolerance of 60.000000
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread filesystem_map INFO Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs']
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread kt_renewer INFO Agent wide credential cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_988
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread agent INFO Using metrics_url_timeout_seconds of 30.000000
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread agent INFO Using task_metrics_timeout_seconds of 5.000000
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread agent INFO Using max_collection_wait_seconds of 10.000000
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread metrics INFO Importing tasktracker metric schema from file /usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/tasktracker/schema.json
>>[11/Aug/2017 15:15:50 +0000] 2409 MainThread ntp_monitor INFO Using timeout of 2.000000
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread dns_names INFO Using timeout of 30.000000
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread __init__ INFO Created DNS monitor.
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread stacks_collection_manager INFO Using max_uncompressed_file_size_bytes: 5242880
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread __init__ INFO Importing metric schema from file /usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/monitor/schema.json
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib64/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUSTEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'SUDO': 'sudo', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Created /run/cloudera-scm-agent/process
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/process to 0751
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Created /run/cloudera-scm-agent/supervisor
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Created /run/cloudera-scm-agent/flood
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/flood to 0751
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Created /run/cloudera-scm-agent/supervisor/include
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent INFO Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
>>[11/Aug/2017 15:15:51 +0000] 2409 MainThread agent ERROR Failed to connect to previous supervisor.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2109, in find_or_start_supervisor
>> self.configure_supervisor_clients()
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.12.0-py2.7.egg/cmf/agent.py", line 2290, in configure_supervisor_clients
>> supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")])
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize
>> Options.realize(self, *arg, **kw)
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize
>> self.process_config()
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config
>> self.process_config_file(do_usage)
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file
>> self.usage(str(msg))
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage
>> self.exit(2)
>>SystemExit: 2
>>[11/Aug/2017 15:15:53 +0000] 2409 Dummy-1 daemonize WARNING Stopping daemon.

the last one node still installing .

thks for help!

Re: CDH 5.12.0 clouder manager agent can not start.

leiyz — Fri, 11 Aug 2017 08:53:21 GMT

hi Michalis.thks for help ,i have found the problem afte umount the tmpfs.thks,regard.