Hello everyone.
I'm going through "Installation path B" for a production cluster. MySQL installed and configured fine, Cloudera Manager starts up fine, and I'm going through the familiar steps of choosing hosts in the cluster on which to have CM install the agent software.
I get to the "Install Step" in CM, and one of the very first thing it does is "Copying installation files" to the other hosts. This process never completes, and the file on the target system just sits at 0 bytes forever until the process times out in CM.
My /etc/hosts file seems correct to me:
CM Server host:
root@bdprodm09:[57]:~> python -c "import socket; print socket.getfqdn(); print socket.gethostbyname(socket.getfqdn())"
bdprodm09.dbhotelcloud.com
172.0.30.1
Agent host:
root@bdprodm10:[37]:~> python -c "import socket; print socket.getfqdn(); print socket.gethostbyname(socket.getfqdn())"
bdprodm10.dbhotelcloud.com
172.0.30.2
I traced the packets with tcpdump between those two IPs, and port 22, and see the traffic, and no errors. But, in the target /tmp/XXX directory, I just get this:
root@bdprodm10:[52]:/tmp/scm_prepare_node.f0oOug56> ls -l
total 0
-rw-r--r-- 1 root root 0 May 2 21:47 scm_prepare_node.sh
This particular time I tried it as root, but I've had the same thing happen with another user account (cminstall) that is set up with passwordless su - root as well:
root@bdprodm10:[57]:/tmp> ls -l scm_prepare_node.uF3EVkb4
total 0
-rw-r--r-- 1 cminstall cminstall 0 May 2 21:41 scm_prepare_node.sh
I tried becoming the 'cminstall' user and manually scp'ing a file from one host to the other, and it worked fine:
[cminstall@bdprodm09 ~]$ scp testfile cminstall@bdprodm10.dbhotelcloud.com:/tmp
cminstall@bdprodm10.dbhotelcloud.com's password:
testfile 100% 33 0.0KB/s 00:00
---
root@bdprodm10:[58]:/tmp> ls -l testfile
-rw-rw-r-- 1 cminstall cminstall 33 May 2 22:07 testfile
The cloudera-scm-server log file doesn't say anything that indicates what is failing:
2016-05-02 22:01:29,243 INFO 164956425@scm-web-36:com.cloudera.server.cmf.node.NodeConfiguratorService: Retrying configurator with id 0
2016-05-02 22:01:29,244 INFO 164956425@scm-web-36:com.cloudera.server.cmf.node.NodeConfiguratorService: Submitted configurator for bdprodm10.dbhotelcloud.com with id 1
2016-05-02 22:01:29,244 INFO NodeConfiguratorThread-6-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: bdprodm10.dbhotelcloud.com: Transitioning from INIT (PT0.001S) to CONNECT
2016-05-02 22:01:29,245 INFO NodeConfiguratorThread-6-1:net.schmizz.sshj.transport.TransportImpl: Client identity string: SSH-2.0-SSHJ_0_14_0
2016-05-02 22:01:29,253 INFO NodeConfiguratorThread-6-1:net.schmizz.sshj.transport.TransportImpl: Server identity string: SSH-2.0-OpenSSH_5.3
2016-05-02 22:01:29,367 INFO NodeConfiguratorThread-6-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: bdprodm10.dbhotelcloud.com: Transitioning from CONNECT (PT0.123S) to AUTHENTICATE
2016-05-02 22:01:29,458 INFO NodeConfiguratorThread-6-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: bdprodm10.dbhotelcloud.com: Transitioning from AUTHENTICATE (PT0.091S) to MAKE_TEMP_DIR
2016-05-02 22:01:29,460 INFO NodeConfiguratorThread-6-1:com.cloudera.server.cmf.node.NodeConfigurator: Executing mktemp -d /tmp/scm_prepare_node.XXXXXXXX on bdprodm10.dbhotelcloud.com
2016-05-02 22:01:29,520 INFO NodeConfiguratorThread-6-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: bdprodm10.dbhotelcloud.com: Transitioning from MAKE_TEMP_DIR (PT0.062S) to COPY_FILES
This is the first time I've had this problem! I've done this install many times in the past, and I'm out of things to check 😞
Hoping someone can shed some light on this one for me!
Thanks very much for your time.
Chris
