Cluster Installation distributing parcel problem

rwesolowski — Wed, 01 Mar 2017 14:58:29 GMT

Hello

I've been trying to install Cloudera 5.10 on Centos 7.x in a VMWare environment. Each time Cloudera Manager gets stuck at Distributed. Only 2 nodes are correct.

Disk space is not a constraint

selinux, ipv6, firewalls are disabled

LOG: (/var/log/cloudera-scm-agent:

[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     SCM Agent Version: 5.10.0
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Agent Protocol Version: 4
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Using Host ID: c0dc62df-1791-496a-a70f-40eea5c1220d
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Using directory: /run/cloudera-scm-agent
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Using supervisor binary path: /usr/lib64/cmf/agent/build/env/bin/supervisord
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Neither verify_cert_file nor verify_cert_dir are configured. Not performing validation of server certificates in HTTPS communication. These options can be configured in this agent's config.ini file to enable certificate validation.
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Agent Logging Level: INFO
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     No command line vars
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Found database jar: /usr/share/java/mysql-connector-java.jar
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if you're not using this database type)
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Found database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
[01/Mar/2017 15:00:07 +0000] 7362 MainThread agent        INFO     Agent starting as pid 7362 user root(0) group root(0).
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Created /run/cloudera-scm-agent/cgroups
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/cgroups to 0751
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found existing subsystem cpuacct at /sys/fs/cgroup/cpu,cpuacct
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found existing subsystem cpu at /sys/fs/cgroup/cpu,cpuacct
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found existing subsystem memory at /sys/fs/cgroup/memory
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found existing subsystem blkio at /sys/fs/cgroup/blkio
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found cgroups subsystem: cpu
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found cgroups subsystem: cpuacct
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found cgroups subsystem: memory
[01/Mar/2017 15:00:09 +0000] 7362 MainThread cgroups      INFO     Found cgroups subsystem: blkio
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Found cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes': 8796093022207, 'default_memory_soft_limit_in_bytes': 8796093022207, 'writable_cgroup_dot_procs': True, 'default_cpu_rt_runtime_us': 950000, 'has_cpu': True, 'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct': True, 'has_blkio': True}
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Setting up supervisord event monitor.
[01/Mar/2017 15:00:09 +0000] 7362 MainThread filesystem_map INFO     Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
[01/Mar/2017 15:00:09 +0000] 7362 MainThread filesystem_map INFO     Using timeout of 2.000000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread filesystem_map INFO     Using join timeout of 0.100000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread filesystem_map INFO     Using tolerance of 60.000000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread filesystem_map INFO     Local filesystem types whitelist: ['ext2', 'ext3', 'ext4', 'xfs']
[01/Mar/2017 15:00:09 +0000] 7362 MainThread kt_renewer   INFO     Agent wide credential cache set to /run/cloudera-scm-agent/krb5cc_cm_agent_0
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Using metrics_url_timeout_seconds of 30.000000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Using task_metrics_timeout_seconds of 5.000000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Using max_collection_wait_seconds of 10.000000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread metrics      INFO     Importing tasktracker metric schema from file /usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/monitor/tasktracker/schema.json
[01/Mar/2017 15:00:09 +0000] 7362 MainThread ntp_monitor INFO     Using timeout of 2.000000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread dns_names    INFO     Using timeout of 30.000000
[01/Mar/2017 15:00:09 +0000] 7362 MainThread __init__     INFO     Created DNS monitor.
[01/Mar/2017 15:00:09 +0000] 7362 MainThread stacks_collection_manager INFO     Using max_uncompressed_file_size_bytes: 5242880
[01/Mar/2017 15:00:09 +0000] 7362 MainThread __init__     INFO     Importing metric schema from file /usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/monitor/schema.json
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Supervised processes will add the following to their environment (in addition to the supervisor's env): {'CDH_PARQUET_HOME': '/usr/lib/parquet', 'JSVC_HOME': '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib64/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE', 'KEYTRUSTEE_KP_HOME': '/usr/share/keytrustee-keyprovider', 'CLOUDERA_ORACLE_CONNECTOR_JAR': '/usr/share/java/oracle-connector-java.jar', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2', 'KEYTRUSTEE_SERVER_HOME': '/usr/lib/keytrustee-server', 'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'HIVE_DEFAULT_XML': '/etc/hive/conf.dist/hive-default.xml', 'CLOUDERA_POSTGRESQL_JDBC_JAR': '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'CDH_KMS_HOME': '/usr/lib/hadoop-kms', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CDH_SQOOP_HOME': '/usr/lib/sqoop', 'WEBHCAT_DEFAULT_XML': '/etc/hive-webhcat/conf.dist/webhcat-default.xml', 'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/lib/hue', 'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar', 'CDH_HBASE_INDEXER_HOME': '/usr/lib/hbase-solr', 'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_SOLR_HOME': '/usr/lib/solr', 'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SENTRY_HOME': '/usr/lib/sentry', 'CDH_CRUNCH_HOME': '/usr/lib/crunch', 'CDH_LLAMA_HOME': '/usr/lib/llama/', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_HIVE_HOME': '/usr/lib/hive', 'ORACLE_HOME': '/usr/share/oracle/instantclient', 'CDH_HCAT_HOME': '/usr/lib/hive-hcatalog', 'CDH_KAFKA_HOME': '/usr/lib/kafka', 'CDH_SPARK_HOME': '/usr/lib/spark', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     To override these variables, use /etc/cloudera-scm-agent/config.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Created /run/cloudera-scm-agent/process
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/process to 0751
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Created /run/cloudera-scm-agent/supervisor
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Created /run/cloudera-scm-agent/flood
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Chowning /run/cloudera-scm-agent/flood to cloudera-scm (996) cloudera-scm (994)
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/flood to 0751
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Created /run/cloudera-scm-agent/supervisor/include
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        INFO     Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
[01/Mar/2017 15:00:09 +0000] 7362 MainThread agent        ERROR    Failed to connect to previous supervisor.
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 2056, in find_or_start_supervisor
    self.configure_supervisor_clients()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 2235, in configure_supervisor_clients
    supervisor_options.realize(args=["-c", os.path.join(self.supervisor_dir, "supervisord.conf")])
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 1599, in realize
    Options.realize(self, *arg, **kw)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 333, in realize
    self.process_config()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 341, in process_config
    self.process_config_file(do_usage)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 376, in process_config_file
    self.usage(str(msg))
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py", line 164, in usage
    self.exit(2)
SystemExit: 2
[01/Mar/2017 15:00:09 +0000] 7362 MainThread tmpfs        INFO     Successfully mounted tmpfs at /run/cloudera-scm-agent/process
[01/Mar/2017 15:00:11 +0000] 7362 MainThread agent        INFO     Trying to connect to newly launched supervisor (Attempt 1)
[01/Mar/2017 15:00:11 +0000] 7362 MainThread agent        INFO     Supervisor version: 3.0, pid: 7391
[01/Mar/2017 15:00:11 +0000] 7362 MainThread agent        INFO     Successfully connected to supervisor
[01/Mar/2017 15:00:11 +0000] 7362 MainThread status_server INFO     Using maximum impala profile bundle size of 1073741824 bytes.
[01/Mar/2017 15:00:11 +0000] 7362 MainThread status_server INFO     Using maximum stacks log bundle size of 1073741824 bytes.
[01/Mar/2017 15:00:11 +0000] 7362 MainThread _cplogging   INFO     [01/Mar/2017:15:00:11] ENGINE Bus STARTING
[01/Mar/2017 15:00:11 +0000] 7362 MainThread _cplogging   INFO     [01/Mar/2017:15:00:11] ENGINE Started monitor thread '_TimeoutMonitor'.
[01/Mar/2017 15:00:11 +0000] 7362 MainThread _cplogging   INFO     [01/Mar/2017:15:00:11] ENGINE Serving on vm01.aposentinel.com:9000
[01/Mar/2017 15:00:11 +0000] 7362 MainThread _cplogging   INFO     [01/Mar/2017:15:00:11] ENGINE Bus STARTED
[01/Mar/2017 15:00:11 +0000] 7362 MainThread __init__     INFO     New monitor: (<cmf.monitor.host.HostMonitor object at 0x1f5af10>,)
[01/Mar/2017 15:00:11 +0000] 7362 MonitorDaemon-Scheduler __init__     INFO     Monitor ready to report: ('HostMonitor',)
[01/Mar/2017 15:00:11 +0000] 7362 MainThread agent        INFO     Setting default socket timeout to 30
[01/Mar/2017 15:00:11 +0000] 7362 Monitor-HostMonitor network_interfaces WARNING Interface 'eth0' reported unknown speed '4294967295'
[01/Mar/2017 15:00:11 +0000] 7362 Monitor-HostMonitor network_interfaces WARNING Interface 'eth0' reported unknown duplex mode '255'
[01/Mar/2017 15:00:12 +0000] 7362 MainThread heartbeat_tracker INFO     HB stats (seconds): num:1 LIFE_MIN:0.24 min:0.24 mean:0.24 max:0.24 LIFE_MAX:0.24
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     CM server guid: 480c885c-27a1-4402-ae51-a415aa0d115e
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     Using parcels directory from server provided value: /opt/cloudera/parcels
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     Created /opt/cloudera/parcels
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     Chowning /opt/cloudera/parcels to root (0) root (0)
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     Chmod'ing /opt/cloudera/parcels to 0755
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     Created /opt/cloudera/parcel-cache
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     Chowning /opt/cloudera/parcel-cache to root (0) root (0)
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        INFO     Chmod'ing /opt/cloudera/parcel-cache to 0755
[01/Mar/2017 15:00:12 +0000] 7362 MainThread parcel       INFO     Agent does create users/groups and apply file permissions
[01/Mar/2017 15:00:12 +0000] 7362 MainThread downloader   INFO     Downloader path: /opt/cloudera/parcel-cache
[01/Mar/2017 15:00:12 +0000] 7362 MainThread parcel_cache INFO     Using /opt/cloudera/parcel-cache for parcel cache
[01/Mar/2017 15:00:12 +0000] 7362 MainThread agent        ERROR    Caught unexpected exception in main loop.
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 710, in __issue_heartbeat
    self._init_after_first_heartbeat_response(resp_data)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 947, in _init_after_first_heartbeat_response
    self.client_configs.load()
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 682, in load
    new_deployed.update(self._lookup_alternatives(fname))
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 432, in _lookup_alternatives
    return self._parse_alternatives(alt_name, out)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 444, in _parse_alternatives
    path, _, _, priority_str = line.rstrip().split(" ")
ValueError: too many values to unpack

LOG /var/log/cloudera/scm-server/

2017-03-01 15:04:24,393 INFO 1215304684@agentServer-3:com.cloudera.server.cmf.AgentProtocolImpl: [DbHost{id=3, hostId=b44317fd-5f50-4019-9266-112580e860e8, hostName=vm02.aposentinel.com}] Added to rack group: /default
2017-03-01 15:04:24,397 INFO 1215304684@agentServer-3:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbHost.name (#3) has changed]}.
2017-03-01 15:04:24,432 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check done. Duration: PT0.003S
2017-03-01 15:04:24,432 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check execution stats: average=0ms, min=0ms, max=0ms.
2017-03-01 15:04:24,507 INFO NodeConfiguratorThread-0-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm02.aposentinel.com: New state is a done state
2017-03-01 15:04:24,507 INFO NodeConfiguratorThread-0-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm02.aposentinel.com: Transitioning from WAIT_FOR_HEARTBEAT (PT4.025S) to COMPLETE
2017-03-01 15:04:24,507 INFO NodeConfiguratorThread-0-1:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-03-01 15:04:31,749 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: Transitioning from PACKAGE_INSTALL cloudera-manager-agent (PT291.417S) to PACKAGE_INSTALL cloudera-manager-daemons
2017-03-01 15:04:32,751 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: Transitioning from PACKAGE_INSTALL cloudera-manager-daemons (PT1.002S) to INSTALL_JCE
2017-03-01 15:04:32,751 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: Transitioning from INSTALL_JCE (PT0S) to AGENT_CONFIGURE
2017-03-01 15:04:32,751 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: Transitioning from AGENT_CONFIGURE (PT0S) to AGENT_START
2017-03-01 15:04:33,422 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: Transitioning from AGENT_START (PT0.671S) to SCRIPT_SUCCESS
2017-03-01 15:04:33,422 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: Transitioning from SCRIPT_SUCCESS (PT0S) to WAIT_FOR_HEARTBEAT
2017-03-01 15:04:37,392 INFO 1215304684@agentServer-3:com.cloudera.server.cmf.AgentProtocolImpl: Setting default rackId for host 8df2aa3d-177f-469d-9154-d5a919dd51e7: /default
2017-03-01 15:04:37,394 INFO 1215304684@agentServer-3:com.cloudera.server.cmf.AgentProtocolImpl: [DbHost{id=4, hostId=8df2aa3d-177f-469d-9154-d5a919dd51e7, hostName=vm04.aposentinel.com}] Added to rack group: /default
2017-03-01 15:04:37,400 INFO 1215304684@agentServer-3:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbHost.name (#4) has changed]}.
2017-03-01 15:04:37,414 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check done. Duration: PT0.003S
2017-03-01 15:04:37,414 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check execution stats: average=0ms, min=0ms, max=0ms.
2017-03-01 15:04:37,457 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: New state is a done state
2017-03-01 15:04:37,457 INFO NodeConfiguratorThread-0-3:com.cloudera.server.cmf.node.NodeConfiguratorProgress: vm04.aposentinel.com: Transitioning from WAIT_FOR_HEARTBEAT (PT4.035S) to COMPLETE
2017-03-01 15:04:37,457 INFO NodeConfiguratorThread-0-3:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-03-01 15:04:56,539 INFO 1748755379@scm-web-4:com.cloudera.server.cmf.components.OperationsManagerImpl: Creating cluster cluster with version CDH 5 and rounded off version CDH 5.0.0
2017-03-01 15:04:56,574 INFO 1748755379@scm-web-4:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbHost.cluster (#1) has changed]}.
2017-03-01 15:04:56,592 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check done. Duration: PT0.004S
2017-03-01 15:04:56,592 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check execution stats: average=0ms, min=0ms, max=0ms.
2017-03-01 15:05:00,614 INFO ParcelUpdateService:com.cloudera.parcel.components.ParcelManagerImpl: Downloading parcels of CDH:5.10.0-1.cdh5.10.0.p0.41 for distros [RHEL7]
2017-03-01 15:05:00,699 INFO ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Preparing to download: https://archive.cloudera.com/cdh5/parcels/5.10/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel
2017-03-01 15:06:41,370 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2017-03-01 15:06:41,391 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2015-03-02T14:06:41.371Z to reap.
2017-03-01 15:06:41,393 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Wizard is active, not reaping scanners or configurators
2017-03-01 15:08:39,803 INFO New I/O worker #19:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: https://archive.cloudera.com/cdh5/parcels/5.10/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel
2017-03-01 15:08:41,790 INFO 696999056@agentServer-5:com.cloudera.server.common.MonitoringThreadPool: agentServer: execution stats: average=14ms, min=0ms, max=207ms.
2017-03-01 15:08:41,791 INFO 696999056@agentServer-5:com.cloudera.server.common.MonitoringThreadPool: agentServer: waiting in queue stats: average=1ms, min=0ms, max=33ms.
2017-03-01 15:08:48,446 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel
2017-03-01 15:08:48,446 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.torrent
2017-03-01 15:08:48,451 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel...
2017-03-01 15:08:48,454 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel with 8 threads (3018 pieces)...
2017-03-01 15:08:49,051 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 10% complete
2017-03-01 15:08:49,672 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 20% complete
2017-03-01 15:08:50,253 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 30% complete
2017-03-01 15:08:50,846 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 40% complete
2017-03-01 15:08:51,463 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 50% complete
2017-03-01 15:08:52,054 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 60% complete
2017-03-01 15:08:52,635 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 70% complete
2017-03-01 15:08:53,226 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 80% complete
2017-03-01 15:08:53,822 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 90% complete
2017-03-01 15:08:54,459 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashed 1 file(s) (1582136490 bytes) in 3018 pieces (3018 expected) in 6002,9ms.
2017-03-01 15:08:54,464 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Single-file torrent information:
2017-03-01 15:08:54,464 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Torrent name: CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel
2017-03-01 15:08:54,464 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Announced at: Seems to be trackerless
2017-03-01 15:08:54,465 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Created on..: Wed Mar 01 15:08:48 CET 2017
2017-03-01 15:08:54,465 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Created by..: cm-server
2017-03-01 15:08:54,465 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Pieces......: 3018 piece(s) (524288 byte(s)/piece)
2017-03-01 15:08:54,465 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Total size..: 1Â 582Â 136Â 490 byte(s)
2017-03-01 15:08:54,493 INFO ParcelUpdateService:com.cloudera.parcel.components.ParcelManagerImpl: Distributing parcel CDH:5.10.0-1.cdh5.10.0.p0.41 on cluster cluster
2017-03-01 15:08:54,591 WARN 271140938@scm-web-12:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.torrent
2017-03-01 15:08:54,594 INFO 271140938@scm-web-12:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.torrent (60527 bytes) to 192.168.114.22 over PT0.002S. Rate: 28,862 MB/s
2017-03-01 15:08:54,631 WARN 271140938@scm-web-12:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.torrent
2017-03-01 15:08:54,632 INFO 271140938@scm-web-12:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.torrent (60527 bytes) to 192.168.114.21 over PT0.001S. Rate: 57,723 MB/s
2017-03-01 15:10:31,281 INFO 1027593378@scm-web-10:com.cloudera.server.web.cmf.ParcelController: Parcel download request: CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel from : 192.168.114.22
2017-03-01 15:10:31,292 INFO 1027593378@scm-web-10:com.cloudera.server.web.cmf.ParcelController: Served parcel piece CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel (1048576 bytes) to 192.168.114.22 over PT0.006S. Rate: 166,667 MB/s
2017-03-01 15:12:10,730 INFO ParcelUpdateService:com.cloudera.parcel.components.ParcelManagerImpl: Activating parcel CDH:5.10.0-1.cdh5.10.0.p0.41 on cluster cluster
2017-03-01 15:12:10,808 INFO ParcelUpdateService:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbCluster.cdhVersionForDb (#1) has changed]}.
2017-03-01 15:12:10,864 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check done. Duration: PT0.011S
2017-03-01 15:12:10,864 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check execution stats: average=0ms, min=0ms, max=0ms.
2017-03-01 15:12:10,922 WARN 1918889447@scm-web-14:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-03-01 15:12:10,922 WARN 1918889447@scm-web-14:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-03-01 15:12:11,093 WARN 1918889447@scm-web-14:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-03-01 15:12:11,093 WARN 1918889447@scm-web-14:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-03-01 15:12:12,138 WARN 1918889447@scm-web-14:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-03-01 15:12:12,138 WARN 1918889447@scm-web-14:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-03-01 15:12:12,484 WARN 1918889447@scm-web-14:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING

The same happened when I was trying to install cloudera by package, only two nodes were correct...
Any help will be appreciated

question Re: Cluster Installation distributing parcel problem in Archives of Support Questions (Read Only)

Cluster Installation distributing parcel problem

Re: Cluster Installation distributing parcel problem

Re: Cluster Installation distributing parcel problem