Support Questions
Find answers, ask questions, and share your expertise

Not able to submit coordinator after public DNS changed

public dns of my master node changed . Now, oozie workflows would not trigger and cannot submit new corrdinators.

ec2-12-123-123-12.ap-south-1.compute.amazonaws.com is the old Public DNS of

 Failed to create deployment directory: HTTPConnectionPool(host='ec2-12-123-123-12.ap-south-1.compute.amazonaws.com', port=50070): Max retries exceeded with url: /webhdfs/v1/user/hue/oozie/deployments/_%24USER_-oozie-%24JOBID-%24TIME?op=GETFILESTATUS&user.name=hue&doas=hue (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc5bcde5ed0>: Failed to establish a new connection: [Errno 110] Connection timed out',)HTTPConnectionPool(host='ec2-12-123-123-12.ap-south-1.compute.amazonaws.com', port=50070): Max retries exceeded with url: /webhdfs/v1/user/hue/oozie/deployments/_%24USER_-oozie-%24JOBID-%24TIME?op=GETFILESTATUS&user.name=hue&doas=hue (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc5bcde5ed0>: Failed to establish a new connection: [Errno 110] Connection timed out',))


Please suggest what values need to be changed.

3 REPLIES 3

Hi @Simran kaur
It seems there are still some hue conf files with pointers to the old DNS record. If you grep for the old DNS address in /etc/hue/conf, do you still see a number of files pointing to this? Mainly hue.ini or sav.* files? You may have to update the ini or remove old sav.* files & restart oozie / hue (# /etc/init.d/hue start) .

Mentor

@Simran kaur

I recently went through the same procedure, it's simple not error prone, it shields you from looking for all the old_DNS_host entries in all the flies.

Here is what to do

Steps

In Ambari Web > Background Operations, stop all pending commands and jobs.

Stop all services.

Stop ambari-server and ambari-agents on all hosts.

# ambari-server stop 
# ambari-agent stop 

Create *.json file with hostnames changes.

Example: host_names_changes.json

{ "cluster1" : 
{ "ec2-12-123-123-12.ap-south-1.compute.amazonaws.com" : "FQDN_New_DNS" } 
} 

where cluster1 is your Actual_cluster_name and "FQDN_old_DNS" : "FQDN_New_DNS"is the hostnames pair in the format "current_host_name" : "new_host_name".

Execute the following command on the ambari-server host:

# ambari-server update-host-names host_names_changes.json 

After the successful end of this action, please update hostnames for all nodes, according to changes that you added to *.json file. If you changed the hostname for the node on which the ambari server resides, then you must update that name for every ambari-agent. In /etc/ambari-agent/conf/ambari-agent.ini, update the "hostname" field to the new hostname for the node on which the ambari-server resides.

Start ambari* services on all hosts.

# ambari-server start 
# ambari-agent start

HTH

Mentor
; ;