The hostname and canonical name for this host are not consistent

[#1]

Hi All,

When I added a new node in cloudera cluster, The node went to bad health and shows error message like "The hostname and canonical name for this host are not consistent when checked from a Java process" Can any one help us to fix this issue.

Thanks,

Sathishkumar M

[#2 · Accepted solution]

If you're running Red Hat or CenOS, ensure the /etc/sysconfig/network file
has the HOSTNAME field with the FQDN, not just the host name.

HOSTNAME=server1.example.com (good)
HOSTNAME=server1 (bad)

Edit this and reboot the host. Just restarting network services might not
work. Then let us know if the error still continues to be logged.

[#3]

The python command should be all one line, or use a "\" to continue on a new line.

For example:

$ python -c "import socket; print socket.getfqdn(); print socket.gethostbyname(socket.getfqdn())"

or
$ python -c "import socket; print socket.getfqdn(); \
print socket.gethostbyname(socket.getfqdn())"

[#4]

Hi ,  I also had the same issue in RHEL 7 . As suggested I modified the "HOSTNAME" attribute in /etc/sysconfig/network file to FQDN. Following that I also modified the hostname to FQDN using the following command:

hostnamectl set-hostname "<your hostname FQDN>" --static 

Followed by this  I restarted cloudera-scm-agent . This worked for me , there were no more critical warnings on hostname and canonical hostname .

[#5]

What am I doing wrong? How can I correct this?
1. DNS Resolution Error
2. Job History Server Web UI going to

"This site can’t be reached
instance-1.c.1345678.internal’s server IP address could not be found."


hostname -f
instance-1.c.12345678.internal

cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.123.0.2 instance-1.12345678.internal instance-1 # Added by Google
123.456.789.012 metadata.google.internal # Added by Google

cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=localhost

python -c "import socket; print socket.getfqdn(); print socket.gethostbyname(socket.getfqdn())";
instance-1.c.12345678.internal
10.123.0.2

ifconfig eth0 | grep "inet addr"
inet addr:10.123.0.2 Bcast:10.123.0.2 Mask:255.255.255.255

[#6]

@Genthandsome,

Hi,

I see you are posting on a lot of different old threads.  In the future, please create a new thread and explain the issue you are seeing like you did here.  Thank you.

If you google "server IP address could not be found" you will find information that helps point you in the right direction about what that means.  You are getting that in Chrome, I presume, so I also guess that you are attempting to connect to the Job History Server UI from a browser that is on a different host than nstance-1.c.12345678.internal

That means your local host (where your browser is installed) cannot resolve the hostname "nstance-1.c.12345678.internal".  You will need to ensure your client host has hosts file or DNS configured to allow it to resolve the host to IP address.

[#7]

error still persists even after restarting

am using centOs 6

[root@instance-1 Admin]# hostname -f
instance-1.c.sacred-evening-197206.internal
[root@instance-1 Admin]# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=instance-1.c.sacred-evening-197206.internal
[root@instance-1 Admin]#

what to do kindly suggest.

[#8]

hi @prabhat10,

Since this is an old thread, we should make sure you are indeed seeing the same issue that is described.

Please let us know what you are doing when you see the problem and what the problem is.

If you are seeing the message that is in the initial comment of this thread, then to get some more insight, I recommend running the DNSTest manually like this on a cluster host:

(make sure java is in your path or specify the full path to the java file)

# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest

The output will be a JSON format file like this:

{"status": "0", "ip": "122.168.100.211", "hostname": "host.example.com", "canonicalname": "host.example.com", "localhostDuration": "4", "canonicalnameDuration": "0" }

This might shed some light on what is going on.  I'd check your hosts file or DNS depending on how your host resolution is configured on that host.

Regards,

Ben

[#9]

hello@bgooley

i have run this command 

# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest

and get this as a output :

[root@instance-1 Admin]# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest
{"status": "0", "ip": "10.142.0.4", "hostname": "instance-1", "canonicalname": "instance-1.c.sacred-evening-197206.internal", "localhostDuration": "4", "canonicalnameDuration": "0" }
[root@instance-1 Admin]# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=instance-1.c.sacred-evening-197206.internal

[root@instance-1 Admin]# java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)
[root@instance-1 Admin]# javac -version
bash: javac: command not found

do i need to run the above command on each hosts and other hosts java is not installed, do i need to instaal it manually. On doing java -version its showing command not found.

[root@instance-1 jdk1.7.0_67-cloudera]# alternatives --display java| grep Current
Current `best' version is /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java.
[root@instance-1 jdk1.7.0_67-cloudera]# ls /usr/java/
jdk1.7.0_67-cloudera
[root@instance-1 jdk1.7.0_67-cloudera]# ls /usr/java/jdk1.7.0_67-cloudera/
bin lib src.zip
COPYRIGHT LICENSE THIRDPARTYLICENSEREADME-JAVAFX.txt
db man THIRDPARTYLICENSEREADME.txt
include README.html
jre release

kindly suggest!!

[#10]

i have run this command on all hosts

# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest

and get this as a output :

[root@instance-1 Admin]# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest
{"status": "0", "ip": "10.142.0.4", "hostname": "instance-1", "canonicalname": "instance-1.c.sacred-evening-197206.internal", "localhostDuration": "4", "canonicalnameDuration": "0" }

[root@instance-2 Admin]# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest
{"status": "0", "ip": "10.142.0.5", "hostname": "instance-2", "canonicalname": "instance-2.c.sacred-evening-197206.internal", "localhostDuration": "5", "canonicalnameDuration": "0" }

[root@instance-3 Admin]# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.c loudera.cmon.agent.DnsTest
{"status": "0", "ip": "10.142.0.2", "hostname": "instance-3", "canonicalname": " instance-3.c.sacred-evening-197206.internal", "localhostDuration": "4", "canonic alnameDuration": "0" }

 but in my /etc/sysconfig/network and /etc/hosts file i have mentioned FQDN then why its showing instance-1, instance-2, instance-3 while running above command.

What to do next kindly suggest.

[#11]

@prabhat10, I'm sorry that I couldn't reply sooner.

The DnsTest command is what Cloudera Manager runs to check your host and canonical host names.

The problem is that your hostname and canonical names are not the same:

"hostname": "instance-1",

"canonicalname": "instance-1.c.sacred-evening-197206.internal

My guess is this has to do with your /etc/hosts file or general DNS resolution

Have you defined the host in /etc/hosts?

If so, make sure it is in the following format:

IP  FQDN

The FQDN name must be first

[#12]

i have run this command on all hosts

# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest

and get this as a output :

[root@instance-1 Admin]# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest
{"status": "0", "ip": "10.142.0.4", "hostname": "instance-1", "canonicalname": "instance-1.c.sacred-evening-197206.internal", "localhostDuration": "4", "canonicalnameDuration": "0" }

[root@instance-2 Admin]# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.cloudera.cmon.agent.DnsTest
{"status": "0", "ip": "10.142.0.5", "hostname": "instance-2", "canonicalname": "instance-2.c.sacred-evening-197206.internal", "localhostDuration": "5", "canonicalnameDuration": "0" }

[root@instance-3 Admin]# java -classpath /usr/share/cmf/lib/agent-5.*.jar com.c loudera.cmon.agent.DnsTest
{"status": "0", "ip": "10.142.0.2", "hostname": "instance-3", "canonicalname": " instance-3.c.sacred-evening-197206.internal", "localhostDuration": "4", "canonic alnameDuration": "0" }

but in my /etc/sysconfig/network and /etc/hosts file i have mentioned FQDN then why its showing instance-1, instance-2, instance-3 while running above command.

What to do next?

Kindly suggest