Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to use config.py

avatar
Contributor

I'm trying to read (and later to set) some of hadoop configuration using configs.py but it fails and I'm unable to figure out what is wrong.

This is what I tried:

[root@hdptst55 ~]# /var/lib/ambari-server/resources/scripts/configs.py -a get -n host_group_1  -u admin -p admin -e yarn-site.xml -k yarn.nodemanager.resource.memory-mb
2018-02-15 16:51:43,771 ERROR File yarn-site.xml doesn't exist or you don't have permissions.
[root@hdptst55 ~]#

The file and the parameters do exists:

[root@hdptst55 ~]# grep -a1 yarn.nodemanager.resource.memory-mb /etc/hadoop/conf/yarn-site.xml
    <property>
      <name>yarn.nodemanager.resource.memory-mb</name>
      <value>5120</value>
[root@hdptst55 ~]#

What is wrong? Are there logs to this tool?

Were can I find some practical usage examples?

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Yuval Smp

First you will need to get the yarn-site config using "action=get" option as following:

From Ambari Server Host. (Please replace the Cluster name with yours)

# /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=get --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json



Example Output "--action=get"

#  /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=get --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json

2018-02-15 22:27:09,474 INFO ### Performing "get" content:
2018-02-15 22:27:09,474 INFO ### to file "/tmp/yarn_site_payload.json"
2018-02-15 22:27:09,600 INFO ### on (Site:yarn-site, Tag:version1517274643601)



Now you will see that there is a file created here "/tmp/yarn_site_payload.json" that you will need to edit based on your requirement and then push it back using "action=set" (suppose i changed "yarn.nodemanager.resource.memory-mb" default value "4608 to "5120" so i will edit this file and will post it back) as following:

# /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json


Example output of "--action=set"

#  /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json

2018-02-15 22:29:38,568 INFO ### Performing "set":
2018-02-15 22:29:38,568 INFO ### from file /tmp/yarn_site_payload.json
2018-02-15 22:29:38,569 INFO ### PUTting file: "/tmp/yarn_site_payload.json"
2018-02-15 22:29:38,569 INFO ### PUTting json into: doSet_version1518733778569710.json
2018-02-15 22:29:38,719 INFO ### NEW Site:yarn-site, Tag:version1518733778569710


Now you should be able to see the changes in the ambari UI. Ambari will show Restart required. Once you restart the affected service you should see that it has updated your "yarn-site.xml" on the respective hosts.

.

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@Yuval Smp

First you will need to get the yarn-site config using "action=get" option as following:

From Ambari Server Host. (Please replace the Cluster name with yours)

# /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=get --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json



Example Output "--action=get"

#  /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=get --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json

2018-02-15 22:27:09,474 INFO ### Performing "get" content:
2018-02-15 22:27:09,474 INFO ### to file "/tmp/yarn_site_payload.json"
2018-02-15 22:27:09,600 INFO ### on (Site:yarn-site, Tag:version1517274643601)



Now you will see that there is a file created here "/tmp/yarn_site_payload.json" that you will need to edit based on your requirement and then push it back using "action=set" (suppose i changed "yarn.nodemanager.resource.memory-mb" default value "4608 to "5120" so i will edit this file and will post it back) as following:

# /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json


Example output of "--action=set"

#  /var/lib/ambari-server/resources/scripts/configs.py --user=admin --password=admin --port=8080 --action=set --host=localhost --cluster=plain_ambari --config-type=yarn-site --file=/tmp/yarn_site_payload.json

2018-02-15 22:29:38,568 INFO ### Performing "set":
2018-02-15 22:29:38,568 INFO ### from file /tmp/yarn_site_payload.json
2018-02-15 22:29:38,569 INFO ### PUTting file: "/tmp/yarn_site_payload.json"
2018-02-15 22:29:38,569 INFO ### PUTting json into: doSet_version1518733778569710.json
2018-02-15 22:29:38,719 INFO ### NEW Site:yarn-site, Tag:version1518733778569710


Now you should be able to see the changes in the ambari UI. Ambari will show Restart required. Once you restart the affected service you should see that it has updated your "yarn-site.xml" on the respective hosts.

.

avatar
Contributor

Thanks @Jay Kumar SenSharma.

Playing with your reply I learned that "--host" is mandatory. In addition, I figured that I had a typo in one of the parameters (-e instead of -c) so it is better to write out the long format and not the abriviation...