question Re: Changing zookeeper client port in Ambari is not reflected for all services in Archives of Support Questions (Read Only)

Changing zookeeper client port in Ambari is not reflected for all services

johnmart — Tue, 19 Jul 2016 21:22:02 GMT

Hi,

I am trying to deploy a HDP cluster via Ambari on a number of VM's that are each running a different version of zookeeper already.

As expected, I am encountering errors that the zookeeper client port (:2181) is already occupied when I try to deploy zookeeper via Ambari.

I tried changing the client port to an unoccupied port in /zoo.cfg in the *Customize Services* of Ambari deployment. Doing this I was able to successfully deploy Zookeeper, but the services that connect to zookeeper are still looking for it at :2181. An example of this is below:

"kafka-broker" : {
     "properties" : { 
	...
	"zookeeper.connect" : "%HOSTGROUP::host_group_1%:2181",
	...
	}
}

I would prefer not to manually change zookeepers client port on every single service that uses zookeeper. Is there an easier alternative?

Re: Changing zookeeper client port in Ambari is not reflected for all services

KuldeepK — Mon, 19 Aug 2019 08:50:35 GMT

@John Martin

You can go to each service configuration in Ambari UI and change the port number.

For example for Kafka, see below screenshot

Re: Changing zookeeper client port in Ambari is not reflected for all services

johnmart — Wed, 20 Jul 2016 03:57:26 GMT

@Kuldeep Kulkarni

Thanks for getting back to me!

I know that I can manually change the port number for each service, I was wondering if there was an alternative to doing that (I don't know much about Zookeeper).

The reason I ask is because I am working on an automated deployment of HDP, and manually changing the port (even if I did so within a Blueprint) gets in the way of that.

Re: Changing zookeeper client port in Ambari is not reflected for all services

KuldeepK — Wed, 20 Jul 2016 04:02:18 GMT

@John Martin - If you are going to use blueprint then I believe mentioning port in the configuration is the only option.

Re: Changing zookeeper client port in Ambari is not reflected for all services

sunile_manjee — Wed, 20 Jul 2016 09:50:30 GMT

@Kuldeep Kulkarni each service configuration can be updated by using API. Take a look here. You can also make changes in the blueprint.

Configuration update involves the following steps:

Identify the config type to update and note the latest version applied. When a config type is updated the whole property set needs to be updated. So copying the values from the latest and updating specific values or adding/removing properties as needed is the easiest option.
Read the cluster resource and note the version of the type you want to update
Read the config type with the tag and note the properties
Edit the properties as needed and then update the config type
- Config update requires creation of a new version (typically current timestamp is a good choice)
- The new version of the config type must be added and applied to the cluster
Restart affected services/components to have the config take effect

You can use APIs or a wrapper script (/var/lib/ambari-server/resources/scripts/configs.sh) to edit configurations.

Edit configuration using APIs (1.4.1/1.2.5)

Verified against releases 1.4.1/1.2.5

Starting 1.4.2/1.4.3 you will have to add -H option to the curl calls. E.g. -H "X-Requested-By: ambari"

1. Find the latest version of the config type that you need to update.

curl -u admin:admin -X GET http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs Sample OUTPUT { "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs", "Clusters" : { "cluster_name" : "CLUSTER_NAME", "version" : "HDP-2.0.6", "desired_configs" : { ... "mapred-site" : { "user" : "admin", "tag" : "version1384716039631" } ... } } }

2. Read the config type with correct tag

curl -u admin:admin "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631" Sample OUTPUT { "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631", "items" : [ { "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631", "tag" : "version1384716039631", "type" : "mapred-site", "Config" : { "cluster_name" : "CLUSTER_NAME" }, "properties" : { ... THESE ARE THE PROPERTY KEY-VALUE PAIRS ... } }] }

3a. Save a new version of the config and apply it (see 3b for doing it using one call)

curl --user admin:admin -i -X POST -d

'{"type": "mapred-site", "tag": "version1384716041120", "properties" : {"mapreduce.admin.map.child.java.opts" : "-Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN",...}}'

http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations curl --user admin:admin -i -X PUT -d '{"Clusters":{"desired_config" : {"type": "mapred-site", "tag": "version1384716041120"}}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME

3b. Save a new version of the config and apply it using one call

curl --user admin:admin -i -X PUT -d '{"Clusters":{"desired_config" : {"type": "mapred-site", "tag": "version1384716041120", "properties" : {...}}}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME

4. Restart all components or services to have the config change take effect E.g. Stop and Start a service

curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Stop HDFS"}, "ServiceInfo": {"state": "INSTALLED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Start HDFS"}, "ServiceInfo": {"state": "STARTED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS

Re: Changing zookeeper client port in Ambari is not reflected for all services

vpoornalingam — Wed, 20 Jul 2016 15:09:50 GMT

@John Martin

Please post it as an idea in HCC / file an enhancement request with Apache / Hortonworks.

Re: Changing zookeeper client port in Ambari is not reflected for all services

KuldeepK — Thu, 21 Jul 2016 04:49:35 GMT

@Sunile Manjee - Thanks for the info, I'm aware that we can do this using API, as @John Martin wanted to do this while installing HDP cluster hence suggested to put in configs section of ambari blueprint.