Please suggest me for intial load of TBs and/or PBs of data from Proudcution cluster to DR cluster. Do you suggest to migrate/replicate data using Cloudera BDR?   Thanks ... View more

Yes, I am talking about  HDFS, Hive, HBase, Impala, Search etc and also cloudera Manager too from one Cloudera Cluster to Another cloudera cluster data size in petabytes. Hope  this clarifies Thanks ... View more

Issue fixed thanks for your continious support @bgooley Appreciate! ... View more

@bgooley    Edge node integration with CM fixed. I can not see Edge Node hosts are visiable under "All hosts" of Cloudera Manager.  When I clicked on Edge node host within Cloudera Manager, I can see heartbeats and everything recorded properly but  "CDH version is none" Now I have to apply gateway roles on Edge nodes so that Hadoop Clients/Cluster configuration files can be deployed  properly. When I am trying to apply role, its failing and error is Mismatched CDH versions. Since Edge node host has CDH version status None where as CM version is CDH 5.14.3, causing issue.  So question is how do I fix CDH version issue on Edge Node.? ... View more

@bgooley After applying the changes under general section of config.ini. Now no error. Only problem is for edge node cloudera version showing none which is hold me to apply any role on edge node. Please suggest monitored_nodev_filesystem_types=nfs,nfs4,tmpfs ... View more

I have marked unchecked to Use TLS Authentication of Agents to Server   and restarted agent on edge node server. However not able to see on CM unders host tab.   This time I have more cleaned cloudera-scm-agent log but with one error   Monitor-HostMonitor throttling_logger ERROR    Could not find local file system for /var/run/cloudera-scm-agent/process   complete log. [20/Sep/2018 18:18:40 +0000] 123357 MainThread tmpfs INFO Successfully umounted tmpfs at /var/run/cloudera-scm-agent/process [20/Sep/2018 18:18:40 +0000] 123357 MainThread tmpfs INFO Successfully mounted tmpfs at /var/run/cloudera-scm-agent/process [20/Sep/2018 18:18:41 +0000] 123357 MainThread agent INFO Trying to connect to newly launched supervisor (Attempt 1) [20/Sep/2018 18:18:41 +0000] 123357 MainThread agent INFO Supervisor version: 3.0, pid: 123382 [20/Sep/2018 18:18:41 +0000] 123357 MainThread agent INFO Successfully connected to supervisor [20/Sep/2018 18:18:41 +0000] 123357 MainThread status_server INFO Using maximum impala profile bundle size of 1073741824 bytes. [20/Sep/2018 18:18:41 +0000] 123357 MainThread status_server INFO Using maximum stacks log bundle size of 1073741824 bytes. [20/Sep/2018 18:18:41 +0000] 123357 MainThread _cplogging INFO [20/Sep/2018:18:18:41] ENGINE Bus STARTING [20/Sep/2018 18:18:41 +0000] 123357 MainThread _cplogging INFO [20/Sep/2018:18:18:41] ENGINE Started monitor thread '_TimeoutMonitor'. [20/Sep/2018 18:18:42 +0000] 123357 MainThread _cplogging INFO [20/Sep/2018:18:18:42] ENGINE Serving on myucbpaabdapp25.cimbmy.cimbdomain.com:9000 [20/Sep/2018 18:18:42 +0000] 123357 MainThread _cplogging INFO [20/Sep/2018:18:18:42] ENGINE Bus STARTED [20/Sep/2018 18:18:42 +0000] 123357 MainThread __init__ INFO New monitor: (<cmf.monitor.host.HostMonitor object at 0x3edb090>,) [20/Sep/2018 18:18:42 +0000] 123357 MonitorDaemon-Scheduler __init__ INFO Monitor ready to report: ('HostMonitor',) [20/Sep/2018 18:18:42 +0000] 123357 MainThread agent INFO Setting default socket timeout to 45 [20/Sep/2018 18:18:42 +0000] 123357 Monitor-HostMonitor network_interfaces INFO NIC iface eth0 doesn't support ETHTOOL (95) [20/Sep/2018 18:18:42 +0000] 123357 Monitor-HostMonitor throttling_logger ERROR Could not find local file system for /var/run/cloudera-scm-agent/process [20/Sep/2018 18:18:42 +0000] 123357 MainThread heartbeat_tracker INFO HB stats (seconds): num:1 LIFE_MIN:0.10 min:0.10 mean:0.10 max:0.10 LIFE_MAX:0.10 [20/Sep/2018 18:18:42 +0000] 123357 MainThread agent INFO CM server guid: e157e5cc-09e9-4196-bac0-d396d5c1a920 [20/Sep/2018 18:18:42 +0000] 123357 MainThread agent INFO Using parcels directory from server provided value: /opt/cloudera/parcels [20/Sep/2018 18:18:42 +0000] 123357 MainThread parcel INFO Agent does create users/groups and apply file permissions [20/Sep/2018 18:18:42 +0000] 123357 MainThread downloader INFO Downloader path: /opt/cloudera/parcel-cache [20/Sep/2018 18:18:42 +0000] 123357 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache [20/Sep/2018 18:18:42 +0000] 123357 MainThread agent INFO Flood daemon (re)start attempt [20/Sep/2018 18:18:42 +0000] 123357 MainThread agent INFO Triggering supervisord update. [20/Sep/2018 18:18:42 +0000] 123357 MainThread downloader ERROR Failed rack peer update: [Errno 111] Connection refused [20/Sep/2018 18:18:42 +0000] 123357 MainThread firehoses INFO Reporting interval updated: 5.0 -> 60 [20/Sep/2018 18:18:42 +0000] 123357 MainThread agent INFO Active parcel list updated; recalculating component info. [20/Sep/2018 18:18:42 +0000] 123357 MainThread throttling_logger INFO Identified java component java8 with full version JAVA_HOME=/usr/java/default java version "1.8.0_171" Java(TM) SE Runtime Environment (build 1.8.0_171-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode) for requested version . [20/Sep/2018 18:19:42 +0000] 123357 MonitorDaemon-Reporter firehoses INFO Creating a connection to the ACTIVITYMONITOR. [20/Sep/2018 18:19:42 +0000] 123357 MonitorDaemon-Reporter firehoses INFO Creating a connection to the SERVICEMONITOR. [20/Sep/2018 18:19:42 +0000] 123357 MonitorDaemon-Reporter firehoses INFO Creating a connection to the HOSTMONITOR. [20/Sep/2018 18:28:44 +0000] 123357 MainThread heartbeat_tracker INFO HB stats (seconds): num:42 LIFE_MIN:0.06 min:0.02 mean:0.06 max:0.09 LIFE_MAX:0.10 ... View more

I have installed agent and jdk from scrath to another edge node and created self signed certificate and imported into trust store. However exact error reproduced in another node agent. I tried to verify further and its showing ssl handshake failed. [root@myucbpaabdapp25 yum.repos.d]# openssl s_client -connect myucbpaabdapp03:7182 -CAfile /opt/cloudera/security/jks/bda.truststore CONNECTED(00000003) depth=0 C = , ST = , L = , O = , OU = , CN = myucbpaabdapp03 verify error:num=18:self signed certificate verify return:1 depth=0 C = , ST = , L = , O = , OU = , CN = myucbpaabdapp03 verify return:1 140215717300040:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184: --- Certificate chain 0 s:/C=/ST=/L=/O=/OU=/CN=myucbpaabdapp03 i:/C=/ST=/L=/O=/OU=/CN= ... View more