Created on 08-10-2016 10:13 AM - edited 09-16-2022 03:33 AM
Hello all --
I am dynamically retrieving configuration files using the Cloudera Manager API.
In particular, I am interested in Yarn's mapred-site.xml.
When downloading from
http://example.com:7180/api/v12/clusters/Cluster 1/services/yarn/roles/yarn-RESOURCEMANAGER-xyz/process/configFiles/mapred-site.xml
I get what looks like a partial mapred-site.xml
<?xml version="1.0" encoding="UTF-8"?> <!--Autogenerated by Cloudera Manager--> <configuration> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>example.com:19888</value> </property> <property> <name>mapreduce.jobhistory.webapp.https.address</name> <value>example.com:19890</value> </property> </configuration>
However, when I download the client configuration zip file from Cloudera Manager:
the mapred-site.xml contained inside the zip file is much larger and appears to contain all configuration values.
Why is there a difference between the two files? How can I get the version of the file that contains the complete configuration from the CM API?
Thanks!
I am using
Version: Cloudera Enterprise 5.7.1
Created 08-11-2016 01:23 AM
Hi atai,
your first CM API call will download the mapred-site.xml configuration file that this specific role instance was started with (in your case Resource Manager). See that the mapred-site.xml config file in/var/run/cloudera-scm-agent/process/xxx-yarn-RESOURCEMANAGER/mapred-site.xml has the same values.
This mapred-site.xml file contains only the configuration values that are relevant to the Resource Manager role.
You want to download the client configuration files for the YARN service instead, using the UI or a call like
# curl -o yarn-clientconfig.zip -u admin:admin http://cmhost:7180/api/v12/clusters/Cluster%201/services/YARN-1/clientConfig
Created 08-11-2016 01:23 AM
Hi atai,
your first CM API call will download the mapred-site.xml configuration file that this specific role instance was started with (in your case Resource Manager). See that the mapred-site.xml config file in/var/run/cloudera-scm-agent/process/xxx-yarn-RESOURCEMANAGER/mapred-site.xml has the same values.
This mapred-site.xml file contains only the configuration values that are relevant to the Resource Manager role.
You want to download the client configuration files for the YARN service instead, using the UI or a call like
# curl -o yarn-clientconfig.zip -u admin:admin http://cmhost:7180/api/v12/clusters/Cluster%201/services/YARN-1/clientConfig
Created 08-11-2016 06:49 AM
Ah, I think I understand.
Thank you for responding.