question Re: Process for moving HDP Services manually in Support Questions

Process for moving HDP Services manually

vnair — Thu, 19 Nov 2015 05:59:44 GMT

We are looking for a documented process for moving services that are currently not supported by Ambari. The Services I am currently looking for information for:

1. HDFS Jornal Node

2. Map Reduce Job History Server

3. Spark History Server

Re: Process for moving HDP Services manually

cnauroth — Thu, 19 Nov 2015 06:07:07 GMT

If there is no existing documentation covering the JournalNode, then here is my recommendation.

Bootstrap the new server by copying the contents of dfs.journalnode.edits.dir from an existing JournalNode.
Start JournalNode on the new server.
Reconfigure the NameNodes to include the new server in dfs.namenode.shared.edits.dir.
Restart standby NN and verify it remains healthy.
Restart active NN and verify it remains healthy.
Reconfigure the NameNodes to remove the old server from dfs.namenode.shared.edits.dir.
Restart standby NN and verify it remains healthy.
Restart active NN and verify it remains healthy.

Some might note that during the copy in step 1, it's possible that additional transactions are being logged concurrently, so the copy might be out-of-date immediately. This is not a problem though. The JournalNode is capable of "catching up" by synchronizing data from other running JournalNodes. In fact, step 1 is really just an optimization of this "catching up".

Re: Process for moving HDP Services manually

vshukla — Thu, 19 Nov 2015 06:53:02 GMT

For Spark, there is no explicit move Spark history server feature in Ambari.

Just use the Ambari REST API to delete Spark component and re-install Spark History server onto a new node.

Re: Process for moving HDP Services manually

vnair — Tue, 24 Nov 2015 12:57:30 GMT

@vshukla can you provide details on REST API to be used ?

Re: Process for moving HDP Services manually

vnair — Tue, 24 Nov 2015 12:57:47 GMT

Answer from Ambari team for MapReduce History Server

Delete Mapreduce History Server if server died

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Service"},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' http://<ambari-server-name>:8080/api/v1/clusters/<cluster-name>/services/MAPREDUCE2

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Component"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://<ambari-server-name>:8080/api/v1/clusters/<cluster-name>/hosts/<old-host-name>/host_components/HISTORYSERVER

curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://<ambari-server-name>:8080/api/v1/clusters/<cluster-name>/services/MAPREDUCE2/components/HISTORYSERVER

Add Mapreduce History Server to New Server

curl -u admin:admin -H "X-Requested-By: ambari" -i -X POST http://<ambari-server-name>:8080/api/v1/clusters/<cluster-name>/services/MAPREDUCE2/components/HISTORYSERVER

curl -u admin:admin -H "X-Requested-By: ambari" -i -X POST -d '{"host_components" : [{"HostRoles":{"component_name":"HISTORYSERVER"}}] }' http://<ambari-server-name>:8080/api/v1/clusters/<cluster-name>/hosts?Hosts/host_name=<new-host-name>;

curl -u admin:admin -H "X-Requested-By: ambari" -i -X PUT -d '{"ServiceInfo": {"state" : "INSTALLED"}}' http://<ambari-server-name>:8080/api/v1/clusters/<cluster-name>/services/MAPREDUCE2

Go to Ambari Web > Services > MapReduce2 > Configs > Advanced, and modify mapreduce.jobhistory.address and mapreduce.jobhistory.webapp.address to point to the new History Server host.

Restart MapReduce2 Service after saving the configuration.

Re: Process for moving HDP Services manually

amcbarnett — Fri, 05 Feb 2016 09:33:02 GMT

Re: Process for moving HDP Services manually

shashant_panwar — Thu, 29 Sep 2016 08:58:12 GMT

@Vinod : Here are the API details:

Delete SPARK Job Server:

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Service"},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/services/SPARK
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Component"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/hosts/<old-host>/host_components/SPARK_JOBHISTORYSERVER
curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/services/SPARK/components/SPARK_JOBHISTORYSERVER

Add SPARK Job Server to New Server

curl -u admin:admin -H "X-Requested-By: ambari" -i -X POST http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/services/SPARK/components/SPARK_JOBHISTORYSERVER
curl -u admin:admin -H "X-Requested-By: ambari" -i -X POST -d '{"host_components" : [{"HostRoles":{"component_name":"SPARK_JOBHISTORYSERVER"}}] }' http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/hosts?Hosts/host_name=<new-host>
curl -u admin:admin -H "X-Requested-By: ambari" -i -X PUT -d '{"ServiceInfo": {"state" : "INSTALLED"}}' http://:8080/api/v1/clusters/clustername/services/SPARK>:8080/api/v1/clusters/clustername/services/SPARK