Process for Moving Journal Nodes from One Host to Another
In case you would like to move JournalNode service to another host, here are the steps to do so: 1. Put HDFS in safemode
su - hdfs -c 'hdfs dfsadmin -fs hdfs://<active node>:8020 -safemode enter'
2. Execute a save namespace of the Active NameNode
su - hdfs -c 'hdfs dfsadmin -fs hdfs://<active node>:8020 -saveNamespace'
3. Stop all services via Ambari
4. Add journal node to the Ambari database through the API call:
export AMBARI_USER=admin
export PASSWORD=admin
export AMBARI_HOST=localhost
export CLUSTER_NAME=cluster
export MOVE_FROM='old-host.abcd.com'
export MOVE_TO='new-host.abcd.com'
# Tell to ambari we want to install this component on new_host
curl -u $AMBARI_USER:$AMBARI_PASSWORD -H "X-Requested-By:ambari" -i -X POST http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/hosts/$MOVE_TO/host_components/JOURNALNODE
OR
curl
-v -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d
'{"host_components" : [{"HostRoles":{"component_name":"JOURNALNODE"}}]
}'
http://<ambari-server>:8080/api/v1/clusters/<clustername>/hosts?Hosts/host_name=<new
journal node hostname>
5. Within the Ambari web UI, click on the "Host" tab to display all the hosts for the cluster. Click on the host in which you just added the JournalNode service to in step 4. On this page you should see the JournalNode component in the "Components" section. Next to the JournalNode label is a drop down button, click it and select the "install" option. A progress bar should show up displaying the progress of the installation.
OR via Curl
# Trigger installation
curl -u $AMBARI_USER:$AMBARI_PASSWORD -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context": "Install JournalNode","query":"HostRoles/component_name.in('JOURNALNODE')"}, "Body":{"HostRoles": {"state": "INSTALLED"}}}' http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/hosts/$MOVE_TO/host_components
6. Once the installation finishes, update the configuration property below within in the Ambari UI within the HDFS section.
<dfs.namenode.shared.edits.dir>
to include the new journal node and exclude the old one
7. Start the new installed journal node
or Via Curl
# Start JournalNode on new host
curl -u $AMBARI_USER:$AMBARI_PASSWORD -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context": "Start JournalNode","query":"HostRoles/component_name.in('JOURNALNODE')"}, "Body":{"HostRoles": {"state": "STARTED"}}}' http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/hosts/$MOVE_TO/host_components/JOURNALNODE
8. Start the rest of the journal nodes
9. Start the active NameNode to refresh the configuration and then stop it.
10. On the previous active node, run
hdfs namenode -initializeSharedEdits -force"
11. Restart the active NameNode
12. On the standby NameNode run
"hdfs namenode -bootstrapStandby"
(it will prompt you to “re-format filesystem in storage directory /hadoop/hdfs/namenode? (Y or N) , answer Y “ )
13. Start DataNodes
14. Start standby NameNode
15. Remove journal node by the following api call
# Stop JournalNode on old host
curl -u $AMBARI_USER:$AMBARI_PASSWORD -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context": "Stop JournalNode","query":"HostRoles/component_name.in('JOURNALNODE')"}, "Body":{"HostRoles": {"state": "INSTALLED"}}}' http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/hosts/$MOVE_FROM/host_components/JOURNALNODE
# Remove the old component
curl -u $AMBARI_USER:$AMBARI_PASSWORD -H "X-Requested-By:ambari" -i -X DELETE http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/hosts/$MOVE_FROM/host_components/JOURNALNODE
16. Start all services (ENSURE THIS IS DONE AND ZOOKEEPER IS UP AND RUNNING OR NEITHER NAMENODE WILL BE ACTIVE)
(Reference support note also)
