Reply
New Contributor
Posts: 5
Registered: ‎09-12-2018

NOT ABLE to add new hosts with CDH6

I met below error while adding new hosts, it suppose to be "host -t PTR " with IP instead of "dev1", "dev1" was the initial hostname when the machine been created.

 

I have changed everything related to hostname to "cdhm.test.com" and restarted the machine and services. But the error insisted. Please help. 

 

 

/tmp/scm_prepare_node.hSFrEINk 
opening logging file descriptor 
Starting installation script...
Acquiring installation lock...
BEGIN flock 4 
END (0) 
Detecting root privileges...
effective UID is 0 
Detecting distribution...
BEGIN grep Tikanga /etc/redhat-release 
END (1) 
BEGIN grep 'Scientific Linux release 5' /etc/redhat-release 
END (1) 
BEGIN grep Santiago /etc/redhat-release 
END (1) 
BEGIN grep 'CentOS Linux release 6' /etc/redhat-release 
END (1) 
BEGIN grep 'CentOS release 6' /etc/redhat-release 
END (1) 
BEGIN grep 'Scientific Linux release 6' /etc/redhat-release 
END (1) 
BEGIN grep Maipo /etc/redhat-release 
END (1) 
BEGIN grep 'CentOS Linux release 7' /etc/redhat-release 
CentOS Linux release 7.3.1611 (Core) 
END (0) 
/etc/redhat-release ==> CentOS 7 
Detecting Cloudera Manager Server...
BEGIN host -t PTR dev1 
Host dev1 not found: 3(NXDOMAIN) 
END (1) 
BEGIN which python 
/usr/bin/python 
END (0) 
BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' lxh1 7182 
Traceback (most recent call last): 
File "<string>", line 1, in <module> 
File "/usr/lib64/python2.7/socket.py", line 224, in meth 
return getattr(self._sock,name)(*args) 
socket.gaierror: [Errno -2] Name or service not known 
END (1) 
could not contact scm server at dev1:7182, giving up 
waiting for rollback request 
detected rollback request 
rolling back installation 
Reverting changes...
rollback started 

 Below are some configurations I made, 

 

[root@cdhm named]# cat /etc/hostname
cdhm.test.com
[root@cdhm named]# cat /etc/hosts
172.***.***.247 cdhm.test.com cdhm
172.***.***.142 cdh1.test.com cdh1
172.***.***.143 cdh2.test.com cdh2
172.***.***.144 cdh3.test.com cdh3
172.***.***.145 cdh4.test.com cdh4
[root@cdhm named]#cat /etc/sysconfig/network
# Created by anaconda
NETWORKING=yes
HOSTNAME=cdhm.test.com
[root@cdhm named]# uname -a
Linux cdhm.test.com 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@cdhm named]# host -v -t A $(hostname)
Trying "cdhm.test.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4781
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;cdhm.test.com.                 IN      A

;; ANSWER SECTION:
cdhm.test.com.          86400   IN      A       172.***.***.247

;; AUTHORITY SECTION:
test.com.               86400   IN      NS      test.com.

;; ADDITIONAL SECTION:
test.com.               86400   IN      A       172.***.***.247

Received 77 bytes from 172.***.***.247#53 in 0 ms
[root@cdhm named]#host -t PTR 172.***.***.247
247.***.***.172.in-addr.arpa domain name pointer ns.test.com.
247.***.***.172.in-addr.arpa domain name pointer cdhm.test.com.
[root@cdhm named]#  host cdhm.test.com
cdhm.test.com has address 172.***.***.247

 

Expert Contributor
Posts: 133
Registered: ‎01-08-2018

Re: NOT ABLE to add new hosts with CDH6

First of all, I have not tested CDH6 yet, so my reply will be based on CDH5 experience. I suppose the log you provided is from cloudera-scm-agent, correct? And this agent was installed before the renaming of hostname. In that case check "server_host" in "/etc/cloudera-scm-agent/config.ini" and change it to the new name.
New Contributor
Posts: 5
Registered: ‎09-12-2018

Re: NOT ABLE to add new hosts with CDH6

[ Edited ]

thanks for the hints. 

 

I have tried that, and restarted scm-server and scm-agent services on the host. but the error is still,

I also found the name used as "SSL_certificate_hostname" in CONFIGS of database "SCM", I changed that to the new server name, but it changed back to "dev1" after I restarted the "cloudera-scm-server" service.

it's really strange. :'(

New Contributor
Posts: 5
Registered: ‎09-12-2018

Re: NOT ABLE to add new hosts with CDH6

 

2018-09-14 14:45:21,189 WARN NodeConfiguratorThread-10-3:com.cloudera.server.cmf.node.NodeConfigurator: 
Command bash -c 'bash /tmp/scm_prepare_node.DGeF0TTG/scm_prepare_node.sh
--server_version 6.0.0 --server_build 530873
--packages /tmp/scm_prepare_node.DGeF0TTG/packages.scm
--always /tmp/scm_prepare_node.DGeF0TTG/always_install.scm
--x86_64 /tmp/scm_prepare_node.DGeF0TTG/x86_64_packages.scm
--certtar /tmp/scm_prepare_node.DGeF0TTG/cert.tar --unlimitedJCE false
--javaInstallStrategy NONE --agentUserMode ROOT
--cm http://cdhm.test.com/cm6
--skipCloudConfig false -h dev1 | tee /tmp/scm_prepare_node.DGeF0TTG/scm_prepare_node.log;
exit ${PIPESTATUS[0]}' on cdh4.test.com finished with exit status 1

more info, I checked out the server log, /var/log/cloudera-scm-server/cloudera-scm-server.log, I found above error and the wrong hostname "dev1" appear after -h option. but I am not albe to find where the scm grab it from.

 

Anyone, please help !!!

 

Posts: 863
Topics: 1
Kudos: 199
Solutions: 107
Registered: ‎04-22-2014

Re: NOT ABLE to add new hosts with CDH6

@HEWITT,

 

Thank you for supplying all that information.  In fact, the last cloudera-scm-server.log text you provided allowed me to look at the CM source code and determine the most probable cause of your situation.

 

In the command args, we see -h dev1 and it so happens that the only way that Cloudera Manager will set the "-h" option is if a configuration parameter in Cloudera Manager is configured.

It seems you have Server SSL Certificate Host Name. set to "dev1".

 

(1)

 

In Cloudera Manager, navigate to:

Administration --> Settings

Search for "Server SSL Certificate Host Name."

 

(2)

 

Remove the value by clicking the blue arrow next to configuration name unless you are attempting to configure the hostname that agents will use to connect to CM so that it matches the Cloudera Manager server certificate.  By default, this is blank.

 

You could also simply specify the true hostname of CM instead of dev1.

 

(3)

 

Restart CM with service cloudera-scm-server restart

 

I hope this solves the mystery and your headache.

New Contributor
Posts: 5
Registered: ‎09-12-2018

Re: NOT ABLE to add new hosts with CDH6

Sorry Buddy, bad news. 

 

I set it as blank, however after I restarted the service cloudera-scm-server, it changed back to "dev1" automatically. 

 

I don't want to format the machine then install everything again, :(

Posts: 863
Topics: 1
Kudos: 199
Solutions: 107
Registered: ‎04-22-2014

Re: NOT ABLE to add new hosts with CDH6

@HEWITT,

 

Dang... that's odd.

 

I don't see how we could be setting that automatically.

 

What worries me here is that if you were able to make the configuration change and then restart, that would imply a successful write of the configuration change to the CM database.  For the configuration to "revert" indicates the save was not successful after all.

 

I would suggest trying again by restoring the value to the default (no value) and restarting CM.

 

If the value comes back this time, try updating the value with the fully-qualified domain name of the Cloudera Manager host (the one used by the agents to heartbeat).

 

Save/restart CM again.

 

If that still reverts to "dev1" please share a screen shot with us and also look at the Cloudera Manager log file to see if there are any messages (errors, warnings) that indicate a problem writing to the database when you are making the configuration changes.

 

 

Announcements