The feature of being able to download client configs after cluster deployment was initially added to Ambari in 1.7.0 version. This feature helps Hadoop admin in following known use cases:
- This can be helpful if Hadoop admin or some other developer wants to setup a client host that is not being managed by ambari.
- Ambari as of now (2.5.0 version) does not support multiple cluster and if user environment has multiple clusters with same set of client hosts that needs to be configured to interact with all clusters, then these hosts cannot be added to any one particular ambari cluster and needs to be managed outside of the ambari. In this scenario being able to download all client configs can be a helpful feature to set up these client hosts.
Further this feature can also be helpful if user wants to create automation scripts for hadoop application running on an ambari managed cluster without using any existing node of the ambari cluster for running automation code. This will require to do setup of the application which will also include hadoop clients set up on automation dedicated hosts.
As stated earlier this feature was added first in Ambari-1.7.0. This feature has been further enhanced with newer APIs in Ambari-2.5.0. Let's go through each supported API related to downloading client configurations.
1. Download client configuration for a service component. (supported since Ambari-1.7.0)
This API downloads base configs for the client component. This means configuration of the Default config group.
Example:
curl --user admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/clusters/c1/services/HDFS/components/HDFS_CLIENT?format=client_config_t... -o output.tar.gz
Downloaded Tarball:
| tar tvf output.tar.gz
drwx------ root/root 0 2017-01-03 21:47 ./
-rw-r--r-- root/root 6639 2017-01-03 21:47 hdfs-site.xml -rw-r--r-- root/root 2526 2017-01-03 21:47 core-site.xml -rw-r--r-- root/root 5740 2017-01-03 21:47 hadoop-env.sh -rw-r--r-- root/root 6764 2017-01-03 21:47 log4j.properties |
UI Interface:
2. Download client configuration for a host component. (supported since Ambari-1.7.0)
This API downloads configs of the client component on a specific host. Note that ambari supports host overrides via config groups feature and so configuration of a client component on a specific host can be different from the configurations of same client components on other hosts. This API downloads configs of the host group that the host component belongs meaning the actual in effect configuration of the client on that host will be downloaded
Example:
curl --user admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/clusters/c1/hosts/c6401.ambari.apache.org/host_components/HDFS_CLIENT?f... -o output.tar.gz
Downloaded Tarball:
| tar tvf output.tar.gz drwx------ root/root 0 2017-01-03 22:16 ./ -rw-r--r-- root/root 6639 2017-01-03 22:16 hdfs-site.xml -rw-r--r-- root/root 2526 2017-01-03 22:16 core-site.xml -rw-r--r-- root/root 5740 2017-01-03 22:16 hadoop-env.sh -rw-r--r-- root/root 6764 2017-01-03 22:16 log4j.properties |
UI Interface:
3. Download client configuration for a service. (supported from Ambari-2.5.0)
This API downloads base configs (Default config group) for all client components of a service.
Example:
curl --user admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/clusters/c1/services/HIVE/components?format=client_config_tar -o output.tar.gz
Downloaded Tarball:
| tar tvf output.tar.gz drwx------ root/root 0 2017-01-03 23:53 HIVE_CLIENT/./ -rw-r--r-- root/root 19226 2017-01-03 23:53 HIVE_CLIENT/hive-site.xml -rw-r--r-- root/root 2148 2017-01-03 23:53 HIVE_CLIENT/hive-env.sh -rw-r--r-- root/root 3050 2017-01-03 23:53 HIVE_CLIENT/hive-log4j.properties -rw-r--r-- root/root 2652 2017-01-03 23:53 HIVE_CLIENT/hive-exec-log4j.properties drwx------ root/root 0 2017-01-03 23:53 HCAT/./ -rw-r--r-- root/root 1275 2017-01-03 23:53 HCAT/hcat-env.sh |
UI Interface:
4. Download client configuration of a host. (supported from Ambari-2.5.0)
This API downloads effective configurations of all client components installed on a specific host.
Example:
curl --user admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/clusters/c1/hosts/c6401.ambari.apache.org/host_components?format=client... -o output.tar.gz
Downloaded Tarball:
| tar tvf output.tar.gz drwx------ root/root 0 2017-01-03 23:42 YARN_CLIENT/./ -rw-r--r-- root/root 2135 2017-01-03 23:42 YARN_CLIENT/capacity-scheduler.xml -rw-r--r-- root/root 5309 2017-01-03 23:42 YARN_CLIENT/yarn-env.sh -rw-r--r-- root/root 2996 2017-01-03 23:42 YARN_CLIENT/core-site.xml -rw-r--r-- root/root 16318 2017-01-03 23:42 YARN_CLIENT/yarn-site.xml -rw-r--r-- root/root 10396 2017-01-03 23:42 YARN_CLIENT/log4j.properties drwx------ root/root 0 2017-01-03 23:42 HDFS_CLIENT/./ -rw-r--r-- root/root 6639 2017-01-03 23:42 HDFS_CLIENT/hdfs-site.xml -rw-r--r-- root/root 2996 2017-01-03 23:42 HDFS_CLIENT/core-site.xml -rw-r--r-- root/root 5740 2017-01-03 23:42 HDFS_CLIENT/hadoop-env.sh -rw-r--r-- root/root 10396 2017-01-03 23:42 HDFS_CLIENT/log4j.properties |
UI Interface:
5. Download client configuration of a cluster. (supported from Ambari-2.5.0)
This API downloads base configurations (Default config group) of all client service components present in a cluster.
Example:
curl --user admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/clusters/c1/components?format=client_config_tar -o output.tar.gz
Downloaded Tarball:
| tar tvf output.tar.gz drwx------ root/root 0 2017-01-03 23:54 ZOOKEEPER_CLIENT/./ -rw-r--r-- root/root 2477 2017-01-03 23:54 ZOOKEEPER_CLIENT/log4j.properties -rw-r--r-- root/root 310 2017-01-03 23:54 ZOOKEEPER_CLIENT/zookeeper-env.sh drwx------ root/root 0 2017-01-03 23:54 YARN_CLIENT/./ -rw-r--r-- root/root 2135 2017-01-03 23:54 YARN_CLIENT/capacity-scheduler.xml -rw-r--r-- root/root 5309 2017-01-03 23:54 YARN_CLIENT/yarn-env.sh -rw-r--r-- root/root 2996 2017-01-03 23:54 YARN_CLIENT/core-site.xml -rw-r--r-- root/root 16318 2017-01-03 23:54 YARN_CLIENT/yarn-site.xml -rw-r--r-- root/root 10396 2017-01-03 23:54 YARN_CLIENT/log4j.properties drwx------ root/root 0 2017-01-03 23:54 HDFS_CLIENT/./ -rw-r--r-- root/root 6639 2017-01-03 23:54 HDFS_CLIENT/hdfs-site.xml -rw-r--r-- root/root 2996 2017-01-03 23:54 HDFS_CLIENT/core-site.xml -rw-r--r-- root/root 5740 2017-01-03 23:54 HDFS_CLIENT/hadoop-env.sh -rw-r--r-- root/root 10396 2017-01-03 23:54 HDFS_CLIENT/log4j.properties drwx------ root/root 0 2017-01-03 23:54 MAPREDUCE2_CLIENT/./ -rw-r--r-- root/root 2996 2017-01-03 23:54 MAPREDUCE2_CLIENT/core-site.xml -rw-r--r-- root/root 868 2017-01-03 23:54 MAPREDUCE2_CLIENT/mapred-env.sh -rw-r--r-- root/root 6754 2017-01-03 23:54 MAPREDUCE2_CLIENT/mapred-site.xml |
UI Interface:
