Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

cloudera-scm-agent start failed

Highlighted

cloudera-scm-agent start failed

Explorer

I've been troubleshooting this issue for awhile before posting it here.

Any help will be appraciated. I'm new to the forum. Thanks.

 

OS: CentOS 6.5 64-bit

Running on VM

CDH 5.0.0 Standard (Express)

 

cloudera-scm-agent fails to start.

I did manually and it is the same issue.

Cloudera Manager and the agent are on the same node.

Other nodes in the cluster with cloudera agent do not have this issue.

 

# service cloudera-scm-agent start

Starting cloudera-scm-agent:                               [FAILED]

 

I look at the logs. Note the cloudera-scm-agent.log is old.

It is telling me that the Phython script has never get to the point generating the log.

In summary, agent fails and there is no log.

Only cloudera-scm-agent.out

 

-rw-r--r--   1 root root 197026 Apr 28 21:46 cloudera-scm-agent.log

-rw-r--r--   1 root root    949 Apr 29 21:21 cloudera-scm-agent.out

-rw-r--r--   1 root root    202 Apr 28 10:45 cmf_listener.log

-rw-r--r--   1 root root  13763 Apr 28 21:46 supervisord.log

-rw-r--r--   1 root root    442 Apr 28 10:45 supervisord.out

 

cloudera-scm-agent.out

 

 cat /var/log/cloudera-scm-agent/cloudera-scm-agent.out

 

/usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha module is deprecated; use the hashlib module instead

  import sha

Traceback (most recent call last):

  File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 42, in <module>

    import cmf.https

  File "/usr/lib64/cmf/agent/src/cmf/https.py", line 7, in <module>

    from M2Crypto import httpslib

  File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/M2Crypto-0.21.1-py2.6-linux-x86_64.egg/M2Crypto/__init__.py", line 29, in <module>

    import DH

  File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/M2Crypto-0.21.1-py2.6-linux-x86_64.egg/M2Crypto/DH.py", line 5, in <module>

    from util import genparam_callback

  File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/M2Crypto-0.21.1-py2.6-linux-x86_64.egg/M2Crypto/util.py", line 15, in <module>

    m2.util_init(UtilError)

SystemError: new style getargs format but argument is not a tuple

 

1- What does this mean?

It sounds like some parameters passed to the script are not right. But why?

 

2- Is there any way to just uninstall cloudera-scm-agent and reinstall it?

All document I went through, it shows how to uninstall Cloudera Manger and agent.

 

Because of this issue, the node in question cannot communicate with the cloudera manager.

 

 

6 REPLIES 6
Highlighted

Re: cloudera-scm-agent start failed

Master Guru
Was the system python installation changed on your VM to a non-default one on this VM?

What version does '/usr/lib64/cmf/agent/build/env/bin/python' command print when you execute it?
What version does the system 'python' command print when you execute it?

Is the OS version ditto here as on the other nodes that do work?
Highlighted

Re: cloudera-scm-agent start failed

Explorer

Harsh,

 

Thank for the follow-up.

I decided to abandon the idea of troublshooting it.

I basically rebuilt VM and reinstalled Cloudera Manager from scratch.

 

I had three nodes. Evereything was installed form the same copy CDH 5.x Express. So I expected that everything is the same version.

On top of it, the two other nodes work well, excepting the first one running with cloudera-scm-server. I start suspecting the CDH 5.x Express has something to do with it. But I cannot confirm it.

 

Highlighted

Re: cloudera-scm-agent start failed

Master Guru
Thanks for following up! The error presented appeared python syntax related, hence the version questions. I've never seen this happen with CM before among our customers or community users. We'll keep a watch out for other such occurrences.
Highlighted

Re: cloudera-scm-agent start failed

New Contributor

I'm having the same issue.

Three nodes already in cluster, fourth one has this problem.

Re: cloudera-scm-agent start failed

Super Collaborator

For us to be able to help we need to know the following:

 

What Linux Distribution / Release level are you on (e.g. RHEL 6.5) 

 

What is the error message you are seeing (is it exact, or similar? Please post it).

 

What version of openSSL is present (for example yum list "openssl*")

 

The m2crypto handles SSL/TLS for the agent, what TLS related settings do you have set in the /etc/cloudera-scm-agent/config.ini ?

 

(you dont need to present the whole file, just the settings you changed within it if you were doing SSL/TLS config on the agent or in CM). 

Highlighted

Re: cloudera-scm-agent start failed

New Contributor

I have the same problem when I try to start the agent on the same server running cloudera manager server:

What Linux Distribution / Release level are you on : CentOS release 6.6 final

 

What is the error message you are seeing (is it exact).

agent logs:
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/'
>>/usr/lib64/cmf/agent/src/cmf/parcel.py:17: DeprecationWarning: the sets module is deprecated
>> from sets import Set
>>[19/Jun/2015 20:10:33 +0000] 5526 MainThread agent INFO SCM Agent Version: 5.4.1
>>[19/Jun/2015 20:10:33 +0000] 5526 MainThread agent INFO Not starting a new session.
>>[19/Jun/2015 20:10:33 +0000] 5526 MainThread agent        ERROR    Could not determine hostname or ip address; proceeding.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 2100, in parse_arguments
>> ip_address = socket.gethostbyname(fqdn)
>>gaierror: [Errno -2] Name or service not known
>>usage: agent.py [-h] [--agent_dir AGENT_DIR] [--lib_dir LIB_DIR]
>> [--orphan_process_directory_staleness_threshold ORPHAN_PROCESS_DIRECTORY_STALENESS_THRESHOLD]
>> [--orphan_process_directory_refresh_interval ORPHAN_PROCESS_DIRECTORY_REFRESH_INTERVAL]
>> [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
>> PACKAGE_DIR [--parcel_dir PARCEL_DIR]
>> [--supervisord_path SUPERVISORD_PATH]
>> [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
>> [--standalone STANDALONE] [--master MASTER]
>> [--environment ENVIRONMENT] [--host_id HOST_ID]
>> [--disable_supervisord_events] --hostname HOSTNAME
>> --ip_address IP_ADDRESS
>> [--reported_hostname REPORTED_HOSTNAME] [--use_tls]
>> [--client_key_file CLIENT_KEY_FILE]
>> [--client_cert_file CLIENT_CERT_FILE]
>> [--verify_cert_file VERIFY_CERT_FILE]
>> [--verify_cert_dir VERIFY_CERT_DIR]
>> [--client_keypw_file CLIENT_KEYPW_FILE]
>> [--max_cert_depth MAX_CERT_DEPTH] [--logfile LOGFILE]
>> [--logdir LOGDIR] [--optional_token] [--clear_agent_dir]
>> [--sudo_command SUDO_COMMAND]
>>agent.py: error: argument --hostname is required
>>/usr/lib64/cmf/agent/src/cmf/parcel.py:17: DeprecationWarning: the sets module is deprecated
>> from sets import Set
>>[19/Jun/2015 20:10:33 +0000] 5526 MainThread agent INFO SCM Agent Version: 5.4.1
>>[19/Jun/2015 20:10:33 +0000] 5526 MainThread agent INFO Not starting a new session.
>>[19/Jun/2015 20:10:33 +0000] 5526 MainThread agent        ERROR    Could not determine hostname or ip address; proceeding.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 2100, in parse_arguments
>> ip_address = socket.gethostbyname(fqdn)
>>gaierror: [Errno -2] Name or service not known

 

What version of openSSL is present (for example yum list "openssl*") 

openssl.x86_64                                   1.0.1e-30.el6.11

 

The m2crypto handles SSL/TLS for the agent, what TLS related settings do you have set in the /etc/cloudera-scm-agent/config.ini ?

 by default, use_tls=0

max_cert_depth=9

 

(you dont need to present the whole file, just the settings you changed within it if you were doing SSL/TLS config on the agent or in CM). 

Don't have an account?
Coming from Hortonworks? Activate your account here