question Re: Problem with cloudera agent in Support Questions

Problem with cloudera agent

shiv003 — Sat, 18 Mar 2017 16:55:14 GMT

I am getting below in cloudera-scm-agent.log. Below are the logs, could someone please have a look and help me to get rid of this error [18/Mar/2017 11:57:26 +0000] 2378 MainThread agent ERROR Failed to connect to previous supervisor. Traceback (most recent call last): File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/agent.py", line 2056, in find_or_start_supervisor self.configure_supervisor_clients() File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/agent.py", line 2235, in configure_supervisor_clients supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")]) File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 1599, in realize Options.realize(self, *arg, **kw) File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 333, in realize self.process_config() File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 341, in process_config self.process_config_file(do_usage) File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 376, in process_config_file self.usage(str(msg)) File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/options.py", line 164, in usage self.exit(2) SystemExit: 2 [18/Mar/2017 11:57:26 +0000] 2378 MainThread tmpfs INFO Successfully mounted tmpfs at /var/run/cloudera-scm-agent/process [18/Mar/2017 11:57:27 +0000] 2378 MainThread agent INFO Trying to connect to newly launched supervisor (Attempt 1) [18/Mar/2017 11:57:27 +0000] 2378 MainThread agent INFO Supervisor version: 3.0, pid: 2455 [18/Mar/2017 11:57:27 +0000] 2378 MainThread agent INFO Successfully connected to supervisor [18/Mar/2017 11:57:27 +0000] 2378 MainThread status_server INFO Using maximum impala profile bundle size of 1073741824 bytes. [18/Mar/2017 11:57:27 +0000] 2378 MainThread status_server INFO Using maximum stacks log bundle size of 1073741824 bytes. [18/Mar/2017 11:57:27 +0000] 2378 MainThread _cplogging INFO [18/Mar/2017:11:57:27] ENGINE Bus STARTING [18/Mar/2017 11:57:27 +0000] 2378 MainThread _cplogging INFO [18/Mar/2017:11:57:27] ENGINE Started monitor thread '_TimeoutMonitor'. [18/Mar/2017 11:57:28 +0000] 2378 MainThread _cplogging INFO [18/Mar/2017:11:57:28] ENGINE Serving on goli001.nr.com:9000 [18/Mar/2017 11:57:28 +0000] 2378 MainThread _cplogging INFO [18/Mar/2017:11:57:28] ENGINE Bus STARTED [18/Mar/2017 11:57:28 +0000] 2378 MainThread __init__ INFO New monitor: (,) [18/Mar/2017 11:57:28 +0000] 2378 MonitorDaemon-Scheduler __init__ INFO Monitor ready to report: ('HostMonitor',) [18/Mar/2017 11:57:28 +0000] 2378 MainThread agent INFO Setting default socket timeout to 30 [18/Mar/2017 11:57:28 +0000] 2378 Monitor-HostMonitor throttling_logger ERROR Could not find local file system for /var/run/cloudera-scm-agent/process [18/Mar/2017 11:57:28 +0000] 2378 MainThread agent ERROR Heartbeating to 192.168.70.9:7182 failed. Traceback (most recent call last): File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/agent.py", line 1358, in _send_heartbeat self.master_port) File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__ self.conn.connect() File "/usr/lib64/python2.6/httplib.py", line 771, in connect self.timeout) File "/usr/lib64/python2.6/socket.py", line 567, in create_connection raise error, msg error: [Errno 111] Connection refused

Re: Problem with cloudera agent

tadeuuuuu — Thu, 20 Apr 2017 15:22:22 GMT

I'm facing the same error:

>> raise error, msg
>>error: [Errno 111] Connection refused
>>[20/Apr/2017 12:10:37 +0000] 992 MainThread heartbeat_tracker INFO HB stats (seconds): num:42 LIFE_MIN:0.00 min:0.00 mean:0.00 max:0.00 LIFE_MAX:0.01
>>[20/Apr/2017 12:11:37 +0000] 992 MainThread agent ERROR Heartbeating to 172.17.0.2:7182 failed.

Any help will be appreciate!

Thanks!

Re: Problem with cloudera agent

bgooley — Thu, 20 Apr 2017 15:26:49 GMT

@tadeuuuuu and @shiv003:

In @shiv003's problem we see that the agent cannot heartbeat to Cloudera Manager due to " [Errno 111] Connection refused". This means that the agent could not reach Cloudera Manager's host on port 7182, likely since Cloudera Manager is not running on that host.

@tadeuuuuu, while you are seeing the same error about failed heartbeating, the cause could be different. Please share the stack trace that accompanies the heartbeat failure so we can review.

Re: Problem with cloudera agent

tadeuuuuu — Thu, 20 Apr 2017 16:42:21 GMT

Hello,

Now i think i got a different error:

>>tar: Exiting with failure status due to previous errors
>>], path: /opt/cloudera/parcels/.flood/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel ]
>>[20/Apr/2017 13:36:08 +0000] 470 MainThread agent ERROR Heartbeating to 172.17.0.2:7182 failed.

I guess everything at my host it's fine, but FYI i'm running this host within a Docker container.

I dont' know what else should i do. Please help-me...it's a challeging for a job 😞

thanks

Re: Problem with cloudera agent

tadeuuuuu — Thu, 20 Apr 2017 16:44:04 GMT

This is my entire log:

/tmp/scm_prepare_node.Xz9uu8V6
using SSH_CLIENT to get the SCM hostname: 172.17.0.1 60534 22
opening logging file descriptor

Iniciando o script de instalação... Adquirindo bloqueio de instalação...

BEGIN flock 4
END (0)

Detectando privilégios raiz...

effective UID is 0

Detectando distribuição...

BEGIN grep Tikanga /etc/redhat-release
END (1)
BEGIN grep 'CentOS release 5' /etc/redhat-release
END (1)
BEGIN grep 'Scientific Linux release 5' /etc/redhat-release
END (1)
BEGIN grep Santiago /etc/redhat-release
END (1)
BEGIN grep 'CentOS Linux release 6' /etc/redhat-release
END (1)
BEGIN grep 'CentOS release 6' /etc/redhat-release
CentOS release 6.6 (Final)
END (0)
/etc/redhat-release ==> CentOS 6

Detectando o Cloudera Manager Server...

BEGIN host -t PTR 172.17.0.1
Host 1.0.17.172.in-addr.arpa. not found: 3(NXDOMAIN)
END (1)
BEGIN which python
/usr/bin/python
END (0)
BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' 172.17.0.1 7182
END (0)
BEGIN which wget
END (1)
which: no wget in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr)
BEGIN wget -qO- -T 1 -t 1 http://169.254.169.254/latest/meta-data/public-hostname && /bin/echo
/tmp/scm_prepare_node.Xz9uu8V6/scm_prepare_node.sh: line 106: wget: command not found
END (127)

Instalando repositórios do pacote...

validating format of repository file /tmp/scm_prepare_node.Xz9uu8V6/repos/rhel6/cloudera-manager.repo
installing repository file /tmp/scm_prepare_node.Xz9uu8V6/repos/rhel6/cloudera-manager.repo
repository file /tmp/scm_prepare_node.Xz9uu8V6/repos/rhel6/cloudera-manager.repo installed

Atualizando metadados do pacote...

BEGIN yum clean all
Loaded plugins: fastestmirror
Repository cloudera-manager is listed more than once in the configuration
Cleaning repos: base cloudera-cdh5 cloudera-manager extras updates
Cleaning up Everything
Cleaning up list of fastest mirrors
END (0)
BEGIN rm -Rf /var/cache/yum/x86_64
END (0)
BEGIN yum makecache
Loaded plugins: fastestmirror
Repository cloudera-manager is listed more than once in the configuration
Metadata Cache Created
END (0)

Instalando o pacote jdk...

BEGIN yum list installed jdk
Loaded plugins: fastestmirror
Installed Packages
jdk.x86_64 2000:1.6.0_31-fcs @Former Member-manager
END (0)
BEGIN echo jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep jdk
jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons
END (0)
BEGIN yum info jdk
Loaded plugins: fastestmirror
Repository cloudera-manager is listed more than once in the configuration
Determining fastest mirrors
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Installed Packages
Name : jdk
Arch : x86_64
Epoch : 2000
Version : 1.6.0_31
Release : fcs
Size : 143 M
Repo : installed
From repo : cloudera-manager
Summary : Java(TM) Platform Standard Edition Development Kit
URL : http://java.sun.com/
License : Copyright (c) 2011, Oracle and/or its affiliates. All rights
: reserved. Also under other license(s) as shown at the Description
: field.
Description : The Java Platform Standard Edition Development Kit (JDK) includes
: both the runtime environment (Java virtual machine, the Java
: platform classes and supporting files) and development tools
: (compilers, debuggers, tool libraries and other tools).
:
: The JDK is a development environment for building applications,
: applets and components that can be deployed with the Java Platform
: Standard Edition Runtime Environment.

END (0)
BEGIN yum -y install jdk.x86_64
Loaded plugins: fastestmirror
Setting up Install Process
Repository cloudera-manager is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Package 2000:jdk-1.6.0_31-fcs.x86_64 already installed and latest version
Nothing to do
END (0)
remote package jdk installed

Instalando o pacote oracle-j2sdk1.7...

BEGIN yum list installed oracle-j2sdk1.7
Loaded plugins: fastestmirror
Installed Packages
oracle-j2sdk1.7.x86_64 1.7.0+update67-1 installed
END (0)
BEGIN echo jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep oracle-j2sdk1.7
jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons
END (0)
BEGIN yum info oracle-j2sdk1.7
Loaded plugins: fastestmirror
Repository cloudera-manager is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Installed Packages
Name : oracle-j2sdk1.7
Arch : x86_64
Version : 1.7.0+update67
Release : 1
Size : 279 M
Repo : installed
Summary : no description given
URL : http://example.com/no-uri-given
License : unknown
Description : no description given

END (0)
BEGIN yum -y install oracle-j2sdk1.7.x86_64
Loaded plugins: fastestmirror
Setting up Install Process
Repository cloudera-manager is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Package oracle-j2sdk1.7-1.7.0+update67-1.x86_64 already installed and latest version
Nothing to do
END (0)
remote package oracle-j2sdk1.7 installed

Instalando o pacote cloudera-manager-agent...

BEGIN yum list installed cloudera-manager-agent
Loaded plugins: fastestmirror
Installed Packages
cloudera-manager-agent.x86_64 5.11.0-1.cm5110.p0.101.el6 @Former Member-manager
END (0)
BEGIN echo jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-agent
jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons
END (0)
BEGIN yum info cloudera-manager-agent
Loaded plugins: fastestmirror
Repository cloudera-manager is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Installed Packages
Name : cloudera-manager-agent
Arch : x86_64
Version : 5.11.0
Release : 1.cm5110.p0.101.el6
Size : 71 M
Repo : installed
From repo : cloudera-manager
Summary : The Cloudera Manager Agent
URL : http://www.cloudera.com
License : Proprietary
Description : The Cloudera Manager Agent.
:
: The Agent is deployed to machines running services managed by
: Cloudera Manager.

END (0)
Version : 5.11.0
Release : 1.cm5110.p0.101.el6
BEGIN yum -y install cloudera-manager-agent
Loaded plugins: fastestmirror
Setting up Install Process
Repository cloudera-manager is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Package cloudera-manager-agent-5.11.0-1.cm5110.p0.101.el6.x86_64 already installed and latest version
Nothing to do
END (0)
remote package cloudera-manager-agent installed

Instalando o pacote cloudera-manager-daemons...

BEGIN yum list installed cloudera-manager-daemons
Loaded plugins: fastestmirror
Installed Packages
cloudera-manager-daemons.x86_64 5.11.0-1.cm5110.p0.101.el6 @Former Member-manager
END (0)
BEGIN echo jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-daemons
END (0)
jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons
BEGIN yum info cloudera-manager-daemons
Loaded plugins: fastestmirror
Repository cloudera-manager is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Installed Packages
Name : cloudera-manager-daemons
Arch : x86_64
Version : 5.11.0
Release : 1.cm5110.p0.101.el6
Size : 789 M
Repo : installed
From repo : cloudera-manager
Summary : Provides daemons for monitoring Hadoop and related tools.
URL : http://www.cloudera.com
License : Proprietary
Description : This package includes daemons for monitoring and managing Hadoop.

END (0)
Version : 5.11.0
Release : 1.cm5110.p0.101.el6
BEGIN yum -y install cloudera-manager-daemons
Loaded plugins: fastestmirror
Setting up Install Process
Repository cloudera-manager is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: mirror.globo.com
* extras: mirror.globo.com
* updates: mirror.globo.com
Package cloudera-manager-daemons-5.11.0-1.cm5110.p0.101.el6.x86_64 already installed and latest version
Nothing to do
END (0)
remote package cloudera-manager-daemons installed

Instalando arquivos de política de Criptografia de intensidade ilimitada.

Installation not requested. Step will be skipped.

Configurando o Cloudera Manager Agent...

BEGIN grep server_host=172.17.0.1 /etc/cloudera-scm-agent/config.ini
END (1)
BEGIN cp /etc/cloudera-scm-agent/config.ini /etc/cloudera-scm-agent/config.ini.orig
END (0)
BEGIN sed -e 's/\(server_host=\).*/\1172.17.0.1/' -i /etc/cloudera-scm-agent/config.ini
END (0)
scm agent configured
BEGIN sed -e 's/#\(USER=.*\)/\1/' -i /etc/default/cloudera-scm-agent
END (0)

Iniciando o Cloudera Manager Agent...

BEGIN /sbin/service cloudera-scm-agent status
cloudera-scm-agent (pid 470) is running...
END (0)
BEGIN /sbin/service cloudera-scm-agent restart
Stopping cloudera-scm-agent: [60G[[0;32m OK [0;39m]
Starting cloudera-scm-agent: could not open session
[60G[[0;31mFAILED[0;39m]
END (1)
agent logs:
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/'
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 17:15:16 +0000] 5199 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 17:15:16 +0000] 5199 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 17:15:16 +0000] 5199 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 18:22:29 +0000] 7360 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 18:22:29 +0000] 7360 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 18:22:29 +0000] 7360 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 18:33:30 +0000] 8074 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 18:33:30 +0000] 8074 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 18:33:30 +0000] 8074 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 18:40:55 +0000] 8589 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 18:40:55 +0000] 8589 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 18:40:55 +0000] 8589 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[20/Apr/2017 13:32:00 +0000] 458 MainThread agent INFO SCM Agent Version: 5.11.0
>>[20/Apr/2017 13:32:00 +0000] 458 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[20/Apr/2017 13:32:00 +0000] 458 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 17:15:16 +0000] 5199 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 17:15:16 +0000] 5199 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 17:15:16 +0000] 5199 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 18:22:29 +0000] 7360 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 18:22:29 +0000] 7360 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 18:22:29 +0000] 7360 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 18:33:30 +0000] 8074 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 18:33:30 +0000] 8074 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 18:33:30 +0000] 8074 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[19/Apr/2017 18:40:55 +0000] 8589 MainThread agent INFO SCM Agent Version: 5.11.0
>>[19/Apr/2017 18:40:55 +0000] 8589 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[19/Apr/2017 18:40:55 +0000] 8589 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
>>/usr/sbin/cmf-agent: line 36: ulimit: max locked memory: cannot modify limit: Operation not permitted
>>[20/Apr/2017 13:32:00 +0000] 458 MainThread agent INFO SCM Agent Version: 5.11.0
>>[20/Apr/2017 13:32:00 +0000] 458 MainThread agent WARNING Expected mode 0751 for /var/run/cloudera-scm-agent but was 0755
>>[20/Apr/2017 13:32:00 +0000] 458 MainThread agent INFO Re-using pre-existing directory: /var/run/cloudera-scm-agent
END (0)
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/'
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-kms/webapps: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-kms: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/crunch/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/crunch: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/parquet/bin: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/parquet/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/parquet: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/avro: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/examples/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/examples: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/assembly/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/assembly: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-hdfs/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-hdfs: Directory renamed before its status could be extracted
>>tar: Exiting with failure status due to previous errors
>>], path: /opt/cloudera/parcels/.flood/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel ]
>>[20/Apr/2017 13:36:08 +0000] 470 MainThread agent ERROR Heartbeating to 172.17.0.2:7182 failed.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.11.0-py2.6.egg/cmf/agent.py", line 1359, in _send_heartbeat
>> self.master_port)
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
>> self.conn.connect()
>> File "/usr/lib64/python2.6/httplib.py", line 720, in connect
>> self.timeout)
>> File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
>> raise error, msg
>>error: [Errno 111] Connection refused
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO Stopping agent...
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO No extant cgroups; unmounting any cgroup roots
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO No processes are being managed; Supervisor will shutdown.
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO Shutting down supervisord, pid 494
>>[20/Apr/2017 13:36:20 +0000] 470 MonitorDaemon-Reporter __init__ INFO Couldn't get supervisord metrics: process no longer exists (pid=494)
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread __init__ INFO waiting for process to terminate...
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread __init__ INFO Successfully killed process with pid 494
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPING
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('cloudera-9', 9000)) shut down
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Stopped thread '_TimeoutMonitor'.
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPED
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPING
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('cloudera-9', 9000)) already shut down
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE No thread running for None.
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPED
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus EXITING
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus EXITED
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread agent INFO Agent exiting; caught signal 15
>>[20/Apr/2017 13:36:20 +0000] 470 Dummy-14 daemonize WARNING Stopping daemon.
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-kms/webapps: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-kms: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/crunch/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/crunch: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/zookeeper: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/parquet/bin: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/parquet/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/parquet: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/avro: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/examples/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/examples: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/assembly/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/assembly: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/spark: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-hdfs/lib: Directory renamed before its status could be extracted
>>tar: CDH-5.11.0-1.cdh5.11.0.p0.34/lib/hadoop-hdfs: Directory renamed before its status could be extracted
>>tar: Exiting with failure status due to previous errors
>>], path: /opt/cloudera/parcels/.flood/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel ]
>>[20/Apr/2017 13:36:08 +0000] 470 MainThread agent ERROR Heartbeating to 172.17.0.2:7182 failed.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.11.0-py2.6.egg/cmf/agent.py", line 1359, in _send_heartbeat
>> self.master_port)
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 469, in __init__
>> self.conn.connect()
>> File "/usr/lib64/python2.6/httplib.py", line 720, in connect
>> self.timeout)
>> File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
>> raise error, msg
>>error: [Errno 111] Connection refused
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO Stopping agent...
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO No extant cgroups; unmounting any cgroup roots
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO No processes are being managed; Supervisor will shutdown.
>>[20/Apr/2017 13:36:19 +0000] 470 MainThread agent INFO Shutting down supervisord, pid 494
>>[20/Apr/2017 13:36:20 +0000] 470 MonitorDaemon-Reporter __init__ INFO Couldn't get supervisord metrics: process no longer exists (pid=494)
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread __init__ INFO waiting for process to terminate...
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread __init__ INFO Successfully killed process with pid 494
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPING
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('cloudera-9', 9000)) shut down
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Stopped thread '_TimeoutMonitor'.
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPED
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPING
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('cloudera-9', 9000)) already shut down
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE No thread running for None.
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus STOPPED
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus EXITING
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread _cplogging INFO [20/Apr/2017:13:36:20] ENGINE Bus EXITED
>>[20/Apr/2017 13:36:20 +0000] 470 MainThread agent INFO Agent exiting; caught signal 15
>>[20/Apr/2017 13:36:20 +0000] 470 Dummy-14 daemonize WARNING Stopping daemon.
END (0)
end of agent logs.
scm agent could not be restarted, giving up
waiting for rollback request

Re: Problem with cloudera agent

tadeuuuuu — Thu, 20 Apr 2017 16:48:48 GMT

At my host that i'm running the Cloudera Manager Server he bring's up this port:

[root@notebook etc]# netstat -putan
Conexões Internet Ativas (servidores e estabelecidas)
Proto Recv-Q Send-Q Endereço Local Endereço Remoto Estado PID/Program name
tcp 0 0 0.0.0.0:7180 0.0.0.0:* OUÇA 3428/java
tcp 0 0 0.0.0.0:7182 0.0.0.0:* OUÇA 3428/java

And then when i run a container to bind the 7182 port it refuses, because this port it's already been in use and i didnt install the agent on this host.