Created 08-27-2018 09:38 AM
public dns of my master node changed . Now, oozie workflows would not trigger and cannot submit new corrdinators.
ec2-12-123-123-12.ap-south-1.compute.amazonaws.com is the old Public DNS of
Failed to create deployment directory: HTTPConnectionPool(host='ec2-12-123-123-12.ap-south-1.compute.amazonaws.com', port=50070): Max retries exceeded with url: /webhdfs/v1/user/hue/oozie/deployments/_%24USER_-oozie-%24JOBID-%24TIME?op=GETFILESTATUS&user.name=hue&doas=hue (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc5bcde5ed0>: Failed to establish a new connection: [Errno 110] Connection timed out',)HTTPConnectionPool(host='ec2-12-123-123-12.ap-south-1.compute.amazonaws.com', port=50070): Max retries exceeded with url: /webhdfs/v1/user/hue/oozie/deployments/_%24USER_-oozie-%24JOBID-%24TIME?op=GETFILESTATUS&user.name=hue&doas=hue (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc5bcde5ed0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
Please suggest what values need to be changed.
Created 08-27-2018 10:07 AM
Hi @Simran kaur
It seems there are still some hue conf files with pointers to the old DNS record. If you grep for the old DNS address in /etc/hue/conf, do you still see a number of files pointing to this? Mainly hue.ini or sav.* files? You may have to update the ini or remove old sav.* files & restart oozie / hue (# /etc/init.d/hue start) .
Created 08-27-2018 10:34 AM
I recently went through the same procedure, it's simple not error prone, it shields you from looking for all the old_DNS_host entries in all the flies.
Here is what to do
Steps
In Ambari Web > Background Operations, stop all pending commands and jobs.
Stop all services.
Stop ambari-server and ambari-agents on all hosts.
# ambari-server stop # ambari-agent stop
Create *.json file with hostnames changes.
Example: host_names_changes.json
{ "cluster1" : { "ec2-12-123-123-12.ap-south-1.compute.amazonaws.com" : "FQDN_New_DNS" } }
where cluster1 is your Actual_cluster_name and "FQDN_old_DNS" : "FQDN_New_DNS"is the hostnames pair in the format "current_host_name" : "new_host_name".
Execute the following command on the ambari-server host:
# ambari-server update-host-names host_names_changes.json
After the successful end of this action, please update hostnames for all nodes, according to changes that you added to *.json file. If you changed the hostname for the node on which the ambari server resides, then you must update that name for every ambari-agent. In /etc/ambari-agent/conf/ambari-agent.ini, update the "hostname" field to the new hostname for the node on which the ambari-server resides.
Start ambari* services on all hosts.
# ambari-server start # ambari-agent start
HTH
Created 08-27-2018 12:15 PM
Here is the link Hortonworks document