Reply
New Contributor
Posts: 4
Registered: ‎08-10-2016
Accepted Solution

Different configuration file from CM API vs client configuration URL

[ Edited ]

Hello all --

 

I am dynamically retrieving configuration files using the Cloudera Manager API.

In particular, I am interested in Yarn's mapred-site.xml.

 

When downloading from  

 

http://example.com:7180/api/v12/clusters/Cluster 1/services/yarn/roles/yarn-RESOURCEMANAGER-xyz/process/configFiles/mapred-site.xml

I get what looks like a partial mapred-site.xml

 

 

<?xml version="1.0" encoding="UTF-8"?>

<!--Autogenerated by Cloudera Manager-->
<configuration>
  <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>example.com:19888</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.https.address</name>
    <value>example.com:19890</value>
  </property>
</configuration>

 

However, when I download the client configuration zip file from Cloudera Manager:

 

 

the mapred-site.xml contained inside the zip file is much larger and appears to contain all configuration values.

 

Why is there a difference between the two files? How can I get the version of the file that contains the complete configuration from the CM API?

 

Thanks!

 

I am using 

Version: Cloudera Enterprise 5.7.1

 

Cloudera Employee
Posts: 232
Registered: ‎01-15-2015

Re: Different configuration file from CM API vs client configuration URL

Hi atai,

 

your first CM API call will download the mapred-site.xml configuration file that this specific role instance was started with (in your case Resource Manager). See that the mapred-site.xml config file in/var/run/cloudera-scm-agent/process/xxx-yarn-RESOURCEMANAGER/mapred-site.xml has the same values.  

This mapred-site.xml file contains only the configuration values that are relevant to the Resource Manager role.

 

You want to download the client configuration files for the YARN service instead, using the UI or a call like

# curl -o yarn-clientconfig.zip -u admin:admin http://cmhost:7180/api/v12/clusters/Cluster%201/services/YARN-1/clientConfig

 

New Contributor
Posts: 4
Registered: ‎08-10-2016

Re: Different configuration file from CM API vs client configuration URL

Ah, I think I understand.

 

Thank you for responding.