Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CM API Call for Config history

CM API Call for Config history

Rising Star

Hi All,

 

Is there an inhouse CM API call to pull out config history details like you can fetch using "history and rollback" tab under configuration of any service in Cloudera Manager.

 

Regards

5 REPLIES 5

Re: CM API Call for Config history

Super Guru

@Prav,

I added your post to an internal Jira that tracks this feature request.  Currently there is no API access to query the configuration history.

 

You may be able to craft something using sql queries, though, if you have direct access to the Cloudera Manager DB and a bit of time to explore the table relationships.

Each configuration change adds a row to the "REVISIONS" table.

There is a corresponding configuration audit record retained in the "*_AUD" tables. 

 

For example, let's say I wanted to look at all updates to search for all updates to YARN since August 5, 2019, I could find this out with:
select * from revisions where timestamp > 1565023808307;

select * from revisions where timestamp > 1565023808307;
 revision_id | optimistic_lock_version | user_id |   timestamp   |                                    message
-------------+-------------------------+---------+---------------+--------------------------------------------------------------------------------
        1551 |                       0 |       1 | 1565032422447 | Updated configurations.
        1552 |                       0 |         | 1565032423362 |
        1553 |                       0 |         | 1565040629791 |
        1554 |                       0 |      -1 | 1565040645136 | Converted configuration for CDH 6.2.0->CDH 6.3.0-1.cdh6.3.0.p0.1279813 Upgrade
        1555 |                       0 |      -1 | 1565040670262 | _INTERNAL_JCEKS_PASSWORD_REVISION_NiFi Registry (host-10-17-100-225)
        1556 |                       0 |      -1 | 1565040694667 | _INTERNAL_JCEKS_PASSWORD_REVISION_Server (host-10-17-100-226)
        1557 |                       0 |      -1 | 1565040719352 | _INTERNAL_JCEKS_PASSWORD_REVISION_NameNode (host-10-17-100-224)
        1558 |                       0 |      -1 | 1565040793772 | _INTERNAL_JCEKS_PASSWORD_REVISION_NiFi Node (host-10-17-100-226)
        1559 |                       0 |      -1 | 1565040869825 | Setting the value of yarn_hadoop_mapreduce_version to 3.0.0-cdh6.3.0

The list of actual changes that were made are retained (for services and roles) in CONFIGS_AUD so I can find out all that were made for YARN service by first finding out the YARN service ID like this:

 

select * from services where service_type like '%YARN%';

scm=> select service_id,service_type from services where service_type like '%YARN%';
 service_id | service_type
------------+--------------
         23 | YARN

 To see all the configuration changes for YARN service, we can search the "configs_aud" table for service_id=23 and the range of revisions returned by the dates, we might do:

scm=> select rev,attr,value,role_id,service_id from configs_aud where service_id = '23' and rev between (select revision_id from revisions where timestamp > 1565023808307 order by revision_id asc limit 1) and (select revision_id from revisions where timestamp > 1565023808307 order by revision_id desc limit 1);
 rev  |             attr              |           value           | role_id | service_id
------+-------------------------------+---------------------------+---------+------------
 1559 | role_jceks_password           | 37lpgwgyc08ph7p5hqyf32o0f |      82 |         23
 1559 | role_jceks_password           | 4hq5dh4t88m745ec25be9rw52 |     103 |         23
 1559 | role_jceks_password           | dnz65pwwgihhswlr62m304cbt |      86 |         23
 1559 | role_jceks_password           | 4pcorg2f1pl5oesqhj49d1cqi |      85 |         23
 1559 | role_jceks_password           | baw183toi5u3236cgb17c995a |      84 |         23
 1559 | role_jceks_password           | 1u56jweql1i5vpdc1cfrqtt5v |      99 |         23
 1559 | yarn_hadoop_mapreduce_version | 3.0.0-cdh6.3.0            |         |         23

 

I'm not that great at SQL, so the above may look horrid to an expert.  The above is a basic way you can try to build a tool to query config history.

Re: CM API Call for Config history

Rising Star

Thanks @bgooley 

 

Not sure I'll be able to integrate that part but thats good to know . Can you please share JIRA number for my reference.

 

Regards

Re: CM API Call for Config history

Rising Star

Hi @bgooley 

While we are discussing this topic, is there a way(An API call) to get service health running on each host?

For ex: Like how Cloudera Manager lets me list hosts from host tab and show services running on a particular host with health status (green, amber or red).

 

Regards

Highlighted

Re: CM API Call for Config history

Super Guru

@Prav,

 

Sure, the internal Cloudera Jira is OPSAPS-31298.

As for the host roles health, that view is derived from a couple different API calls.

This will return a list of roles for a host:

https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/apidocs/resource_HostsResource.html#resour...

For example:

 

<a href="<a href="https://cm_host.example.com:7183/api/v33/hosts/35b008c0-e3e0-4d0b-9c4b-f4522e01567b?view=full" target="_blank">https://cm_host.example.com:7183/api/v33/hosts/35b008c0-e3e0-4d0b-9c4b-f4522e01567b?view=full</a>" target="_blank"><a href="https://cm_host.example.com:7183/api/v33/hosts/35b008c0-e3e0-4d0b-9c4b-f4522e01567b?view=full</a" target="_blank">https://cm_host.example.com:7183/api/v33/hosts/35b008c0-e3e0-4d0b-9c4b-f4522e01567b?view=full</a</a>>

 

Look at the rolerefs section

To see the health of one of the roles, I could take this:

"clusterName" : "cluster",
    "serviceName" : "yarn",
    "roleName" : "yarn-NODEMANAGER-b7751cbb5e13234c9416f5946bc7f2ee"
  },

And run this query:

 

<a href="<a href="https://cm_host.example.com:7183/api/v33/clusters/cluster/services/yarn/roles/yarn-NODEMANAGER-b7751cbb5e13234c9416f5946bc7f2ee" target="_blank">https://cm_host.example.com:7183/api/v33/clusters/cluster/services/yarn/roles/yarn-NODEMANAGER-b7751cbb5e13234c9416f5946bc7f2ee</a>" target="_blank"><a href="https://cm_host.example.com:7183/api/v33/clusters/cluster/services/yarn/roles/yarn-NODEMANAGER-b7751cbb5e13234c9416f5946bc7f2ee</a" target="_blank">https://cm_host.example.com:7183/api/v33/clusters/cluster/services/yarn/roles/yarn-NODEMANAGER-b7751cbb5e13234c9416f5946bc7f2ee</a</a>>

 

You can get the general role health in the "healthSummary"

Using REST, python, or java, you could iterate over the roles listed in the host's rolerefs section and use the second API call to pull the health for each.

Re: CM API Call for Config history

Super Guru