Member since
05-22-2017
15
Posts
6
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4984 | 02-21-2018 10:22 PM |
10-26-2018
06:01 AM
Hello, Can it be possible to set the alerts for long running jobs? Thank you Bhanu
... View more
05-29-2018
08:31 PM
Hi, Michalis I hit the same situation with non residential gateway host. In my case, the script told, Service: [HDFS] contains role type [GATEWAY] - configuration [key: role_config_suppression_cdh_version_validator - value: true]
Affected instance hostname: cdh-host-x // ipAddress: 10.x.x.x
Service: [YARN] contains role type [GATEWAY] - configuration [key: role_config_suppression_cdh_version_validator - value: true]
Affected instance hostname: cdh-host-x // ipAddress: 10.x.x.x
Service: [HBASE] contains role type [GATEWAY] - configuration [key: role_config_suppression_cdh_version_validator - value: true]
Affected instance hostname: cdh-host-x // ipAddress: 10.x.x.x
Service: [HIVE] contains role type [GATEWAY] - configuration [key: role_config_suppression_cdh_version_validator - value: true]
Affected instance hostname: cdh-host-x // ipAddress: 10.x.x.x
Service: [SPARK2_ON_YARN] contains role type [GATEWAY] - configuration [key: role_config_suppression_cdh_version_validator - value: true]
Affected instance hostname: cdh-host-x // ipAddress: 10.x.x.x I once removed the host mentioned from the cluster and cloudera manager. At this point, mgmt service began working as ordinary. Then, re-added and applied host template to deploy cdh components again. After fixing package deployment, I ran the script above again that showed no list 🙂 Thanks a lot.
... View more
03-08-2018
01:30 AM
One quick solution is to add application host to cluster and give (only) HDFS Gateway role to it. Then you can use NameNode Nameservice name as uri e.g. hdfs://nameservice1/. This could be done with CM. https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cm_mc_adding_hosts.html https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cm_mc_managing_roles.html Pros: - Can manage configuration centrally. Cons: - Host goes under CM management i.e. get monitored by agent, be capable of getting assigned other roles and so on. - Files are distributed e.g. under /opt/cloudera which consume disk space (take log space into consideration also). - Some ports - 9000 and 9001 are used by agent. These might cross administrative boundary that please carefully consider and make a plan beforehand.
... View more
03-01-2018
12:52 AM
I noticed that from monitoring alert list, I can set Free Space Monitoring Percentage Thresholds. I couldn't find other path to these setting though. It may help if made clear where those adjustables are exposed I think.
... View more
02-21-2018
10:22 PM
as a quick solution, added auto recovery code to our tool, i.e. check datanode role status after cluster startup completion and run start command for each stopped datanode role. goes like this Welcome to
_ __
| | /| / /______ ____ ___
| |/ |/ / __/ _ `/ _ \(_-<
|__/|__/_/ \_,_/ .__/___/ version 1.0.2
/_/
starting hdfs-DATANODE-ac7041aa53e984590b7d2e27a66ae6ed
starting hdfs-DATANODE-c3a2e16a1c264acbf2b3a8cd036c8abd
starting hdfs-DATANODE-59269ad5f41a6f45c24d9971f1e45660
starting hdfs-DATANODE-47a0b595206a0616ff011606dff76d0f
waiting 30 sec.
HDFS health checks [0]
+-----------------------------------+---------------+
| NAME | SUMMARY |
+-----------------------------------+---------------+
|HDFS_BLOCKS_WITH_CORRUPT_REPLICAS | GOOD |
|HDFS_CANARY_HEALTH | BAD |
|HDFS_DATA_NODES_HEALTHY | CONCERNING |
|HDFS_FAILOVER_CONTROLLERS_HEALTHY | GOOD |
|HDFS_FREE_SPACE_REMAINING | GOOD |
|HDFS_HA_NAMENODE_HEALTH | GOOD |
|HDFS_MISSING_BLOCKS | GOOD |
|HDFS_UNDER_REPLICATED_BLOCKS | GOOD |
+-----------------------------------+---------------+
... View more
06-02-2017
07:23 AM
Thanks a lot. I have encountered the same problem while upgrading from CDH5.10.0 to CDH5.11.0. Management services (including Navigator) where not able to start. I have followed your instructions and after restart of cloudera agent, the mgmt services were able to start.
... View more