Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Python cm_api to refresh and deploy the client configuration not working completely

Solved Go to solution

Python cm_api to refresh and deploy the client configuration not working completely

Contributor

Hi Cloudera Gurus!

 

I am working with AWS and  Lambda functions written in Python working with automatization. 

I can add datanodes properly now but, always, after the deploy the configuration for HDFS nd YARN, I need to "refresh the cluster" and I don't know how. I tried the following:

 

cms.hosts_start_roles(hosts)

if refresh == True:
    cms.hosts_start_roles(hosts)
    print "Deploying client configuration for HDFS and YARN"
    yarn.deploy_client_config()
    time.sleep(30)
    hdfs.refresh()
    time.sleep(30)
    print "Refresh HDFS nd YARN"
    hdfs.deploy_client_config()
    time.sleep(30)
    yarn.refresh()
    time.sleep(30)
    refresh = False
    print "Finish"

I don't know what I am missing but for HDFS is working all perfectly, but not for YARN. Image attached:

Screen Shot 2017-05-04 at 17.29.18.png

 

 

Any ideas what I am missing in my python code?

I am really confused. 

 

 

Thanks all

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Python cm_api to refresh and deploy the client configuration not working completely

Super Collaborator

You might want to try refreshing the ResourceManager (RM) from the UI.

 

That would be going to CM> YARN> Instances> RM> [Actions]> Refresh RM;

With the API this would be Commands>Refresh see [0] and python [1]

"For YARN services, this command should be executed on ResourceManager roles. It refreshes the role's queue and node information."

 

Let me know if this works for you.

 

[0] https://cloudera.github.io/cm_api/apidocs/v16/path__clusters_-clusterName-_services_-serviceName-_ro...

[1] https://github.com/cloudera/cm_api/blob/master/python/src/cm_api/endpoints/services.py#L824-L831

4 REPLIES 4

Re: Python cm_api to refresh and deploy the client configuration not working completely

Contributor

any help here?

Using CDH 5.10.0

Re: Python cm_api to refresh and deploy the client configuration not working completely

Contributor

No ideas about what could be happening here?

 

REgards

 

Re: Python cm_api to refresh and deploy the client configuration not working completely

Super Collaborator

You might want to try refreshing the ResourceManager (RM) from the UI.

 

That would be going to CM> YARN> Instances> RM> [Actions]> Refresh RM;

With the API this would be Commands>Refresh see [0] and python [1]

"For YARN services, this command should be executed on ResourceManager roles. It refreshes the role's queue and node information."

 

Let me know if this works for you.

 

[0] https://cloudera.github.io/cm_api/apidocs/v16/path__clusters_-clusterName-_services_-serviceName-_ro...

[1] https://github.com/cloudera/cm_api/blob/master/python/src/cm_api/endpoints/services.py#L824-L831

Highlighted

Re: Python cm_api to refresh and deploy the client configuration not working completely

Contributor
you are right.

Working now. Thanks for your help!