Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Super Mentor

###Finding Stale Configs

- We can use the Ambari APIs to find the stale configurations. (Notice: ?HostRoles/stale_configs=true)

http://erie1.example.com:8080/api/v1/clusters/ErieCluster/host_components?HostRoles/stale_configs=tr...

- If there is no stale configuration present in the cluster then we might see the items as empty "[]". Output may be as following:

{
  "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/host_components?HostRoles/stale_configs=true",
  "items" : [ ]
}

.

- Now i intentionally made some changes to the "Advanced core-site.xml" from ambari and added the following properties to it, Just to see if it changes the stale configuration list.

hadoop.proxyuser.joy.hosts=*
hadoop.proxyuser.joy.groups=*

.

- As we made some changes and the affected components are not yet restarted hence we will see that there are few stale configuration present in the cluster then we might see the output something like following (Notice: "stale_configs" : true )

{
  "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/host_components?HostRoles/stale_configs=true",
  "items" : [
    {
      "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/hosts/erie3.example.com/host_components/APP_TIMELINE_SERVER",
      "HostRoles" : {
        "cluster_name" : "ErieCluster",
        "component_name" : "APP_TIMELINE_SERVER",
        "host_name" : "erie3.example.com",
        "stale_configs" : true
      },
      "host" : {
        "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/hosts/erie3.example.com"
      }
    },
    {
      "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/hosts/erie1.example.com/host_components/DATANODE",
      "HostRoles" : {
        "cluster_name" : "ErieCluster",
        "component_name" : "DATANODE",
        "host_name" : "erie1.example.com",
        "stale_configs" : true
      },
      "host" : {
        "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/hosts/erie1.example.com"
      }
    },
    .
    .
    .======== SKIPPED some part of the output ========
    .
    {
      "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/hosts/erie2.example.com/host_components/ZKFC",
      "HostRoles" : {
        "cluster_name" : "ErieCluster",
        "component_name" : "ZKFC",
        "host_name" : "erie2.example.com",
        "stale_configs" : true
      },
      "host" : {
        "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/hosts/erie2.example.com"
      }
    }
  ]
}

.

###How to fix the Stale Config using Ambari APIs?

## Until Ambari 2.6.x ##

Now we will see how we can fix the mentioned stale configuration changes. We will need to restart the affected components in order to refresh the stale configurations. We can use the following ambari APIs (it is applicable for Ambari 2.4 and above)

# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d  '{"RequestInfo":{"command":"RESTART","context":"Restart all required services","operation_level":"host_component"},"Requests/resource_filters":[{"hosts_predicate":"HostRoles/stale_configs=true"}]}' http://erie1.example.com:8080/api/v1/clusters/ErieCluster/requests

.

Example output, when the request will be successfully accepted by ambari:

OUTPUT
=======
{
  "href" : "http://erie1.example.com:8080/api/v1/clusters/ErieCluster/requests/427",
  "Requests" : {
    "id" : 427,
    "status" : "Accepted"
  }
}

.

NOTE: The ["command":"RESTART"] will cause the affected services to be restarted completely (example NameNodes & DataNodes) hence before running the above command make sure that you do it in a maintenance windows (downtime expected)

## From Ambari 2.7.x onwards ##

If you are using Ambari 2.7.0 then in the API call users should also pass the "HostRoles/cluster_name=XXXXX" cluster_name property as it is mandatory.

# curl -u admin:admin -H "X-Requested-By: ambari"  -X POST -d '{"RequestInfo":{"command":"RESTART","context":"Restart all required services","operation_level":"host_component"},"Requests/resource_filters":[{"hosts_predicate":"HostRoles/stale_configs=true&HostRoles/cluster_name=DemoCluster"}]}' http://latest1.example.com:8080/api/v1/clusters/DemoCluster/requests

OUTPUT
========
{
  "href" : "http://latest1.example.com:8080/api/v1/clusters/DemoCluster/requests/314",
  "Requests" : {
    "id" : 314,
    "status" : "Accepted"
  }
}

.

2,079 Views
Comments

The API call you provided does not only start the components with stale configs, but actually all components. It has two typos, this one worked for me:

curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d  '{"RequestInfo":{"command":"RESTART","context":"Restart all required services","operation_level":"host_components"},"Requests/resource_filters":[{"hosts_predicate":"HostRoles/stale_configs=true"}]}' http://erie1.example.com:8080/api/v1/clusters/ErieCluster/requests
New Contributor

@Jay Kumar SenSharma

Nice article. There is small correction in API to fix stale configurations:

"operation_level":"host_components" --> "operation_level":"host_component" 
and
{"hosts_predicate":"HostRoles/stale_configs:true"} --> {"hosts_predicate":"HostRoles/stale_configs=true"}

Here is the corrected API:

curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d  '{"RequestInfo":{"command":"RESTART","context":"Restart all required services","operation_level":"host_component"},"Requests/resource_filters":[{"hosts_predicate":"HostRoles/stale_configs=true"}]}' http://erie1.example.com:8080/api/v1/clusters/ErieCluster/requests
Super Mentor

@Dharmesh Jain

Thanks for sharing the findings. It was really a very keen observation. Wonderful !!!

I have incorporated the changes as you suggested above.

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎12-21-2016 09:51 AM
Updated by:
 
Contributors
Top Kudoed Authors