Install error in Windows Azure: Installation failed. Failed to receive heartbeat from agent.

[#1]

Install error in Windows Azure VM: Installation failed. Failed to receive heartbeat from agent.

Windows Azure Virtual Machine OS: Ubuntu 1204 LTS

CDH: Version 5.0 beta 2

>>[17/Feb/2014 01:25:00 +0000] 2587 MainThread agent INFO To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
>>[17/Feb/2014 01:25:00 +0000] 2587 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent/process
>>[17/Feb/2014 01:25:00 +0000] 2587 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor
>>[17/Feb/2014 01:25:00 +0000] 2587 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor/include
>>[17/Feb/2014 01:25:00 +0000] 2587 MainThread agent INFO Connecting to previous supervisor: agent-1329-1392600082.
>>[17/Feb/2014 01:25:00 +0000] 2587 MainThread _cplogging INFO [17/Feb/2014:01:25:00] ENGINE Bus STARTING
>>[17/Feb/2014 01:25:00 +0000] 2587 MainThread _cplogging INFO [17/Feb/2014:01:25:00] ENGINE Started monitor thread '_TimeoutMonitor'.
>>[17/Feb/2014 01:25:00 +0000] 2587 HTTPServer Thread-2 _cplogging ERROR [17/Feb/2014:01:25:00] ENGINE Error in HTTP server: shutting down
>>Traceback (most recent call last):
>> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/process/servers.py", line 187, in _start_http_thread
>> self.httpserver.start()
>> File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1825, in start
>> raise socket.error(msg)
>>error: No socket could be created on ('hd2.cloudapp.net', 9000) -- [Errno 99] Cannot assign requested address
>>
>>[17/Feb/2014 01:25:00 +0000] 2587 HTTPServer Thread-2 _cplogging INFO [17/Feb/2014:01:25:00] ENGINE Bus STOPPING
>>[17/Feb/2014 01:25:00 +0000] 2587 HTTPServer Thread-2 _cplogging INFO [17/Feb/2014:01:25:00] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('hd2.cloudapp.net', 9000)) already shut down
>>[17/Feb/2014 01:25:00 +0000] 2587 HTTPServer Thread-2 _cplogging INFO [17/Feb/2014:01:25:00] ENGINE Stopped thread '_TimeoutMonitor'.
>>[17/Feb/2014 01:25:00 +0000] 2587 HTTPServer Thread-2 _cplogging INFO [17/Feb/2014:01:25:00] ENGINE Bus STOPPED
>>[17/Feb/2014 01:25:00 +0000] 2587 HTTPServer Thread-2 _cplogging INFO [17/Feb/2014:01:25:00] ENGINE Bus EXITING
>>[17/Feb/2014 01:25:00 +0000] 2587 HTTPServer Thread-2 _cplogging INFO [17/Feb/2014:01:25:00] ENGINE Bus EXITED
END (0)
end of agent logs.
scm agent started
Installation script completed successfully.
all done
closing logging file descriptor

[#2 · Accepted solution]

As per knowledge and experience heartbeat must be on NameNode port.

Kindly check this link which explains how to use sharedlib in oozie.
http://blog.cloudera.com/blog/2012/12/how-to-use-the-sharelib-in-apache-oozie/

[#3 · Accepted solution]

By default namenode port is 8020. You need to check your configuration and check fs.default.name property in hdfs-site.xml to get your exact port configured for heartbeat.

Default port information for different services you can get in below link:
http://blog.cloudera.com/blog/2009/08/hadoop-default-ports-quick-reference/

[#4]

Hey,

Try to check your hosts configuration. Try digg more with hosts file configuration, I am sure you will get solution for your issue.

Some more information on same problem is mentioned in one of the cloudera community forum. Kindly check forum link : http://community.cloudera.com/t5/Cloudera-Manager-Installation/Failed-to-receive-heartbeat-from-agen...

[#5]

You are right. The installation error is about hosts configuration.

The Windows Azure VM has two ip, the one is Public virtual IP (VIP) address, another is Internal IP address.

In the /etc/hosts, I set local VM host as Internal IP address, and remote VM host as Public virtual IP (VIP). The installation is success.

But start Ooize service occurred error in the stop of Installing Oozie ShareLib in HDFS.

Would you tell me which port the heartbeat is? I think I should add the port to Windows Azure VM Endpoints.

[#6 · Accepted solution]

As per knowledge and experience heartbeat must be on NameNode port.

Kindly check this link which explains how to use sharedlib in oozie.
http://blog.cloudera.com/blog/2012/12/how-to-use-the-sharelib-in-apache-oozie/

[#7]

Thanks ChiraqPatadia.

Would you tell me what the heartbeat port number is?

[#8 · Accepted solution]

By default namenode port is 8020. You need to check your configuration and check fs.default.name property in hdfs-site.xml to get your exact port configured for heartbeat.

Default port information for different services you can get in below link:
http://blog.cloudera.com/blog/2009/08/hadoop-default-ports-quick-reference/

[#9 · Accepted solution]

In addition to the blog link, the ciomplete catalog of ports supporting the integration is here

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Instal...

This breaks down based on CDH component, CM, etc.

[#10]

Thanks Tgrayson.

[#11]

Thanks for your help.

[#12]

And I think all the hosts ip address must be Internal IP address. Did you test installing cdh in Windows Azure Virtual Machine service?