Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Determine Ambari Host and Cluster Name from command line for configs.sh

Solved Go to solution
Highlighted

Determine Ambari Host and Cluster Name from command line for configs.sh

New Contributor

Hi,

I'm writing a script that will set up the necessary classpath on a Spark cluster managed by Ambari.

Originally, I had a script that would just edit /etc/spark/conf/spark-defaults.conf directly. However, as discussed here, that was bad solution since Ambari regularly overwrites that file with whatever configuration is registered with Ambari.

Therefore, I am now using the configs.sh script as described here to make these adjustments.

I am looking for a way to determine the AMBARI_HOST and CLUSTER_NAME arguments via the command line. Here is what I have so far:

#!/usr/bin/env bash

ambari_host=  # TODO: some command to find this out
cluster_name= # TODO: some command to find this out
config_types=("spark.executor.extraClassPath" "spark.driver.extraClassPath")
phoenix_jar="$(ls /usr/hdp/current/phoenix-client/phoenix-*-client-spark.jar)"
sqljdbc_jar="$(ls /usr/share/java/sqljdbc*.jar)"
jars="$phoenix_jar:$sqljdbc_jar"

for config_type in "${config_types[@]}"; do
    /var/lib/ambari-server/resources/scripts/configs.sh \
	set "$ambari_host" "$cluster_name" spark-defaults "$config_type" "$jars"
done

I can elaborate on anything that's unclear. Thanks in advance!

Edit: thank you everyone for the answers! I wish I could accept all of them.

For posterity, here's what I ended up with:

#!/usr/bin/env bash

ambari_user="$1"
ambari_password="$2"
ambari_port=8080

ambari_host="$(/opt/hostname_scripts/hostname.sh)"
cluster_name="$(curl -u ${ambari_user}:${ambari_password} -i -H 'X-Requested-By: ambari'  http://$ambari_host:$ambari_port/api/v1/clusters | sed -n 's/.*"cluster_name" : "\([^\"]*\)".*/\1/p')"

config_types=("spark.executor.extraClassPath" "spark.driver.extraClassPath")

phoenix_jar="$(ls /usr/hdp/current/phoenix-client/phoenix-*-client-spark.jar)"
sqljdbc_jar="$(ls /usr/share/java/sqljdbc*.jar)"
jars="$phoenix_jar:$sqljdbc_jar"

for config_type in "${config_types[@]}"; do
    /var/lib/ambari-server/resources/scripts/configs.sh -u "$ambari_user" -p "$ambari_password" -port "$ambari_port" set "$ambari_host" "$cluster_name" spark-defaults "$config_type" "$jars" > /dev/null
done 

This was also a valuable resource.

[1] https://community.hortonworks.com/questions/43587/spark-defulatsconf-constantly-being-overwritten.ht...

[2] https://cwiki.apache.org/confluence/display/AMBARI/Modify+configurations#Modifyconfigurations-Editco...

[3] http://lecluster.delaurent.com/one-shot-backup-all-config-files-with-ambari-api/

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Determine Ambari Host and Cluster Name from command line for configs.sh

@Zach Kirsch here is how I usually detect the cluster name using parsing the json output via sed

export SERVICE=ZEPPELIN
export PASSWORD=admin
export AMBARI_HOST=localhost

#detect name of cluster
output=`curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari'  http://$AMBARI_HOST:8080/api/v1/clusters`

CLUSTER=`echo $output | sed -n 's/.*"cluster_name" : "\([^\"]*\)".*/\1/p'`

echo $CLUSTER
4 REPLIES 4

Re: Determine Ambari Host and Cluster Name from command line for configs.sh

Rising Star

@Zach Kirsch

1) I don't know what you mean by AMBARI_HOST? Is it where your ambari server is installed? From any of the agent hosts, you can find the server host from : /etc/ambari-agent/conf/ambari-agent.ini.

Look for the entry [server] hostname=$SERVER_HOST

2) Are you ok using API?

You can get the cluster name by running :

curl --user admin:admin http://$AMBARI_HOST:8080/api/v1/clusters/

This will return you a response of the type :

{ "href" : "http://$AMBARI_HOST:8080/api/v1/clusters/", "items" : [ { "href" : "http://$AMBARI_HOST:8080/api/v1/clusters/clustername", "Clusters" : { "cluster_name" : "clustername", "version" : "version" } } ]}

You may then extract the cluster name.

Hope this helps!

Re: Determine Ambari Host and Cluster Name from command line for configs.sh

Expert Contributor
@Zach Kirsch

1) To find out cluster_name, you can either use the api curl --user admin:admin http://$AMBARI_SERVER:8080/api/v1/clusters/ as mentioned by Shreya,

Or you can query the database for table 'clusters' as below :

select cluster_name from clusters;

2) If you are looking to find out ambari server host, then as @sbhat mentioned, you can grep ambari-agent.ini. Below command would do :

grep "hostname" /etc/ambari-agent/conf/ambari-agent.ini

But if you are looking for the host on which particular spark component is installed, then please use this api :

curl --user admin:admin http://$AMBARI_SERVER:8080/api/v1/clusters/<cluster_name>/services/SPARK/<component_name>

From the response, parse the json for the field 'host_components'

Re: Determine Ambari Host and Cluster Name from command line for configs.sh

@Zach Kirsch here is how I usually detect the cluster name using parsing the json output via sed

export SERVICE=ZEPPELIN
export PASSWORD=admin
export AMBARI_HOST=localhost

#detect name of cluster
output=`curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari'  http://$AMBARI_HOST:8080/api/v1/clusters`

CLUSTER=`echo $output | sed -n 's/.*"cluster_name" : "\([^\"]*\)".*/\1/p'`

echo $CLUSTER

Re: Determine Ambari Host and Cluster Name from command line for configs.sh

Super Guru
@Zach Kirsch

You can get Ambari server hostname from ambari-agent.ini file.

example:

[root@kknode3 ~]# cat /etc/ambari-agent/conf/ambari-agent.ini |grep -i host
hostname = kknode1.openstacklocal
Don't have an account?
Coming from Hortonworks? Activate your account here