CM_API fails executing host_install

[#1]

Ubuntu 16.04 LTS

Python 2 (and pip)

Docker version 1.13.1 (go 1.6.2 git commit 092cba3)

CM_API version 19

I am trying to automate the entire installation procedure (with Docker on my local development environment), which I expect it should be the same as following the web ui wizard. The example I use is based on https://github.com/cloudera/cm_api/blob/master/python/examples/cluster_set_up.py. However, even with the sample python script i.e. cluster_set_up.py (with corresponded information modified for my local environment). It fails with error

    Installing hosts. This might take a while.
    cm_host_install failed: Failed to complete installation.

I am able to install cluster and services (kafka, and zookeeper) through web ui wizard without a problem. So I am not sure why this doesn't work through cm_api. And the error message is not clear indicating what goes wrong. Any place to debug and fix this problem?

[#2]

@tjford,

Check out the Cloudera Manager logs for any issues.

Also, in Cloudera Manager, navigate to:

http://cm_host:7180/cmf/commands/commands

Find the failed add hosts command there and see if you can discover any details there about the problem.

NOTE:  The "cm_host_install failed" message is from the example "cluster_set_up.py" and occurs when the host add command fails here:

cm.host_install(host_username, host_list, password=host_password, cm_repo_url=cm_repo_url)

Make sure you have configured your hosts' username and password.  The defaults are "root:cloudera"

host_username = "root"
host_password = "cloudera"

If you haven't changed those variable values then that's likely the cause as the user/pass would not be correct for your hosts.

[#3]

The message logged in http://cm_host:7180/cmf/commands/commands is a bit obscure. It merely shows

    Failed to complete installation on host 1 of 1.

        Failed to complete installation on host 172.17.0.2.

Where ip i.e 172.17.0.2 corresponds to cm_host in cluster_set_up.py 

Click 'Download' button (at the same level as Status line) shows that the actually problem is Connection refused.

What service are expected for this communication? And which port should I open so the api host_install call can talk to that service?

Thanks

Update:

Executing curl command

    curl -u "admin:<default ui pass>" -X POST -d '{"userName": "root", "password": "<default pass>", "hostNames": ["localhost"]}' -i -H content-type:application/json http://localhost:7180/api/v7/cm/commands/hostInstall

returns

    HTTP/1.1 200 OK
    Expires: Thu, 01-Jan-1970 00:00:00 GMT
    Set-Cookie: CLOUDERA_MANAGER_SESSIONID=uw1r3bw5imx5140fu8714bzgb;Path=/;HttpOnly
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 14:53:21 GMT
    Transfer-Encoding: chunked
    Server: Jetty(6.1.26.cloudera.4)

    {
        "id" : 16,
        "name" : "GlobalHostInstall",
        "startTime" : "2018-07-17T14:53:21.925Z",
        "active" : true,
        "children" : {
            "items" : [ ]
        }
    }

 But checking url localhost:7180/cmf/command/30/details (through web ui), it's still complaing 'connection refused'

[#4]

Checking api page - https://cloudera.github.io/cm_api/apidocs/v19/path__cm_commands_hostInstall.html

The command 'host_install' looks like for installing cloudera manager agent.

Does this mean it's not necessary to execute host_install(...) function if cloudera-scm-agent is already presented (installed through package manager)?

[#5 · Accepted solution]

@tjford,

Yup, if you have already installed the "cloudera-manager-agent" and "cloudera-manager-daemons" packages, configured the agent's config.ini with the server_host as your CM host, and started it (so that it is heartbeating to Cloudera Manager) then you can probably just add it to the cluster using this:

https://cloudera.github.io/cm_api/apidocs/v5.15.0/path__clusters_-clusterName-_hosts.html

[#6]

Thanks I suppose this is solved!