Support Questions

Find answers, ask questions, and share your expertise

Different configuration file from CM API vs client configuration URL

avatar
Explorer

Hello all --

 

I am dynamically retrieving configuration files using the Cloudera Manager API.

In particular, I am interested in Yarn's mapred-site.xml.

 

When downloading from  

 

http://example.com:7180/api/v12/clusters/Cluster 1/services/yarn/roles/yarn-RESOURCEMANAGER-xyz/process/configFiles/mapred-site.xml

I get what looks like a partial mapred-site.xml

 

 

<?xml version="1.0" encoding="UTF-8"?>

<!--Autogenerated by Cloudera Manager-->
<configuration>
  <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>example.com:19888</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.https.address</name>
    <value>example.com:19890</value>
  </property>
</configuration>

 

However, when I download the client configuration zip file from Cloudera Manager:

 

 

the mapred-site.xml contained inside the zip file is much larger and appears to contain all configuration values.

 

Why is there a difference between the two files? How can I get the version of the file that contains the complete configuration from the CM API?

 

Thanks!

 

I am using 

Version: Cloudera Enterprise 5.7.1

 

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hi atai,

 

your first CM API call will download the mapred-site.xml configuration file that this specific role instance was started with (in your case Resource Manager). See that the mapred-site.xml config file in/var/run/cloudera-scm-agent/process/xxx-yarn-RESOURCEMANAGER/mapred-site.xml has the same values.  

This mapred-site.xml file contains only the configuration values that are relevant to the Resource Manager role.

 

You want to download the client configuration files for the YARN service instead, using the UI or a call like

# curl -o yarn-clientconfig.zip -u admin:admin http://cmhost:7180/api/v12/clusters/Cluster%201/services/YARN-1/clientConfig

 

View solution in original post

2 REPLIES 2

avatar
Super Collaborator

Hi atai,

 

your first CM API call will download the mapred-site.xml configuration file that this specific role instance was started with (in your case Resource Manager). See that the mapred-site.xml config file in/var/run/cloudera-scm-agent/process/xxx-yarn-RESOURCEMANAGER/mapred-site.xml has the same values.  

This mapred-site.xml file contains only the configuration values that are relevant to the Resource Manager role.

 

You want to download the client configuration files for the YARN service instead, using the UI or a call like

# curl -o yarn-clientconfig.zip -u admin:admin http://cmhost:7180/api/v12/clusters/Cluster%201/services/YARN-1/clientConfig

 

avatar
Explorer

Ah, I think I understand.

 

Thank you for responding.