Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Cloudera Director 2.0 - Cluster Health Not Available

avatar
Contributor

Greetings,

 

I recently deployed Cloudera Director 2.0 and successfully pushed out a CDH 5 AWS cluster.

 

I am running into a wierd issue where Cloudera Director does not know the state of the cluster.

 

http://i.imgur.com/4fxmk1o.png 

 

While the cluster shows perfectly healthy in Cloudera Manager.

 

http://i.imgur.com/gBSb6Vc.png

 

I verified that Cloudera Director can SSH into the other instances without issue.

 

Any thoughts?

 

I see no errors in the Cloudera Director application logs other than an INFO level log line "RequestMappingInfo did not match any include patterns"

 

The logs: http://pastebin.com/35D7tspS  

1 ACCEPTED SOLUTION

avatar
Rising Star

Hi,

 

Thanks for the information you provided. Since you have changed the Cloudera Manager credentials, it is possible to update Director to use the same, which should restore access, and have the health show up again.

 

On your Director, navigate to the status page for the deployment, i.e. for the CM. On the right there is a drop down menu, click on "Update Cloudera Manager Credentials". This should pop up a form where you can save the new credentials for Director. After a bit the health for the cluster will show up...

 

Hope this helps,

Jayita

View solution in original post

6 REPLIES 6

avatar
Rising Star

Hello,

 

Are you seeing the unknown health consistently, or is it intermittent? If it's consistent, is it possible the credentials were changed on Cloudera Manager? If the password changed, it's possible Director cannot login and fetch the required metrics.

 

If that's not the case, could you also check the status of the cluster via the API? The example URL would be:

http://<directorhost>:7189/api/v4/environments/<environment_name>/deployments/<demployment_name>/clusters/<cluster_name>/ This will list the specific status for every service. 

 

Thanks,

Jayita

avatar
Contributor

Greetings,

 

The issue is persistant. We did change the default admin password in Cloudera Manager (necessary to keep the environment secure, or anyone in the company could login). 

 

Is there a way I can make Director aware of a password change? It was also unable to see the status before the psw change. 

avatar
Contributor

I have run the API command you suggested that dumps the cluster status. This was the result.

 

[ec2-user@ip-10-170-0-201 ~]$ curl --user admin:<redacted> http://10.170.0.201:7189/api/v4/environments/New%20Platform/deployments/CM01/clusters/
[ "Cluster01" ][ec2-user@ip-10-170-0-201 ~]$ curl --user admin:CDH_g0liv3 http://10.170.0.201:7189/api/v4/environments/New%20Platform/deployments/CM01/clusters/Cluster01
{
"name" : "Cluster01",
"instances" : [ {
"ipAddress" : "10.170.12.140",
"properties" : {
"subnetId" : "subnet-1e1f6f47",
"ebsOptimized" : "false",
"virtualizationType" : "hvm",
"imageId" : "ami-414b7271",
"instanceType" : "m4.large",
"keyName" : "NewPlatformRoot",
"publicDnsName" : "",
"sourceDestCheck" : "true",
"availabilityZone" : "us-west-2c",
"privateIpAddress" : "10.170.12.140",
"rootDeviceType" : "ebs",
"launchTime" : "Fri Jan 22 09:59:22 EST 2016",
"instanceId" : "i-709e08aa",
"placementGroupName" : "",
"hypervisor" : "xen",
"spot" : "false",
"vpcId" : "vpc-1b47767e",
"rootDeviceName" : "/dev/sda1",
"privateDnsName" : "ip-10-170-12-140.us-west-2.compute.internal",
"securityGroups" : "launch-wizard-1(sg-fe2aca99)",
"architecture" : "x86_64"
},
"virtualInstance" : {
"id" : "cce0aec6-3eaf-4126-8ccc-63b1e4af4d9b",
"template" : {
"name" : "ClouderaManager",
"type" : "m4.large",
"image" : "ami-414b7271",
"bootstrapScript" : "",
"config" : {
"subnetId" : "subnet-1e1f6f47",
"rootVolumeType" : "gp2",
"instanceNamePrefix" : "director",
"rootVolumeSizeGB" : "50",
"useSpotInstances" : "false",
"securityGroupsIds" : "sg-fe2aca99"
},
"tags" : {
"Name" : "NewPlatform-ClouderaManager"
},
"normalizeInstance" : true
}
},
"capabilities" : {
"operatingSystemType" : "REDHAT_COMPATIBLE",
"operatingSystemVersion" : "REDHAT_COMPATIBLE_6",
"virtualizationType" : "HARDWARE_ASSISTED",
"pythonVersion" : "2.6.6",
"packageManager" : "YUM",
"passwordlessSudoEnabled" : true,
"selinuxEnabled" : true,
"iptablesEnabled" : false
},
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
}
}, {
"ipAddress" : "10.170.1.109",
"properties" : {
"subnetId" : "subnet-1e1f6f47",
"ebsOptimized" : "false",
"virtualizationType" : "hvm",
"imageId" : "ami-414b7271",
"instanceType" : "m4.large",
"keyName" : "NewPlatformRoot",
"publicDnsName" : "",
"sourceDestCheck" : "true",
"availabilityZone" : "us-west-2c",
"privateIpAddress" : "10.170.1.109",
"rootDeviceType" : "ebs",
"launchTime" : "Fri Jan 22 09:59:22 EST 2016",
"instanceId" : "i-7f9e08a5",
"placementGroupName" : "",
"hypervisor" : "xen",
"spot" : "false",
"vpcId" : "vpc-1b47767e",
"rootDeviceName" : "/dev/sda1",
"privateDnsName" : "ip-10-170-1-109.us-west-2.compute.internal",
"securityGroups" : "launch-wizard-1(sg-fe2aca99)",
"architecture" : "x86_64"
},
"virtualInstance" : {
"id" : "ccdc7ab5-f6ef-4f46-be65-1f4664b42433",
"template" : {
"name" : "ClouderaManager",
"type" : "m4.large",
"image" : "ami-414b7271",
"bootstrapScript" : "",
"config" : {
"subnetId" : "subnet-1e1f6f47",
"rootVolumeType" : "gp2",
"instanceNamePrefix" : "director",
"rootVolumeSizeGB" : "50",
"useSpotInstances" : "false",
"securityGroupsIds" : "sg-fe2aca99"
},
"tags" : {
"Name" : "NewPlatform-ClouderaManager"
},
"normalizeInstance" : true
}
},
"capabilities" : {
"operatingSystemType" : "REDHAT_COMPATIBLE",
"operatingSystemVersion" : "REDHAT_COMPATIBLE_6",
"virtualizationType" : "HARDWARE_ASSISTED",
"pythonVersion" : "2.6.6",
"packageManager" : "YUM",
"passwordlessSudoEnabled" : true,
"selinuxEnabled" : true,
"iptablesEnabled" : false
},
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
}
}, {
"ipAddress" : "10.170.1.108",
"properties" : {
"subnetId" : "subnet-1e1f6f47",
"ebsOptimized" : "false",
"virtualizationType" : "hvm",
"imageId" : "ami-414b7271",
"instanceType" : "m4.large",
"keyName" : "NewPlatformRoot",
"publicDnsName" : "",
"sourceDestCheck" : "true",
"availabilityZone" : "us-west-2c",
"privateIpAddress" : "10.170.1.108",
"rootDeviceType" : "ebs",
"launchTime" : "Fri Jan 22 09:59:22 EST 2016",
"instanceId" : "i-7c9e08a6",
"placementGroupName" : "",
"hypervisor" : "xen",
"spot" : "false",
"vpcId" : "vpc-1b47767e",
"rootDeviceName" : "/dev/sda1",
"privateDnsName" : "ip-10-170-1-108.us-west-2.compute.internal",
"securityGroups" : "launch-wizard-1(sg-fe2aca99)",
"architecture" : "x86_64"
},
"virtualInstance" : {
"id" : "2ff16e18-24e7-4352-92f8-b74e292c4d90",
"template" : {
"name" : "ClouderaManager",
"type" : "m4.large",
"image" : "ami-414b7271",
"bootstrapScript" : "",
"config" : {
"subnetId" : "subnet-1e1f6f47",
"rootVolumeType" : "gp2",
"instanceNamePrefix" : "director",
"rootVolumeSizeGB" : "50",
"useSpotInstances" : "false",
"securityGroupsIds" : "sg-fe2aca99"
},
"tags" : {
"Name" : "NewPlatform-ClouderaManager"
},
"normalizeInstance" : true
}
},
"capabilities" : {
"operatingSystemType" : "REDHAT_COMPATIBLE",
"operatingSystemVersion" : "REDHAT_COMPATIBLE_6",
"virtualizationType" : "HARDWARE_ASSISTED",
"pythonVersion" : "2.6.6",
"packageManager" : "YUM",
"passwordlessSudoEnabled" : true,
"selinuxEnabled" : true,
"iptablesEnabled" : false
},
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
}
}, {
"ipAddress" : "10.170.1.110",
"properties" : {
"subnetId" : "subnet-1e1f6f47",
"ebsOptimized" : "false",
"virtualizationType" : "hvm",
"imageId" : "ami-414b7271",
"instanceType" : "m4.large",
"keyName" : "NewPlatformRoot",
"publicDnsName" : "",
"sourceDestCheck" : "true",
"availabilityZone" : "us-west-2c",
"privateIpAddress" : "10.170.1.110",
"rootDeviceType" : "ebs",
"launchTime" : "Fri Jan 22 09:59:22 EST 2016",
"instanceId" : "i-7d9e08a7",
"placementGroupName" : "",
"hypervisor" : "xen",
"spot" : "false",
"vpcId" : "vpc-1b47767e",
"rootDeviceName" : "/dev/sda1",
"privateDnsName" : "ip-10-170-1-110.us-west-2.compute.internal",
"securityGroups" : "launch-wizard-1(sg-fe2aca99)",
"architecture" : "x86_64"
},
"virtualInstance" : {
"id" : "9bdde564-c490-45ad-864b-172f3907ee0a",
"template" : {
"name" : "ClouderaManager",
"type" : "m4.large",
"image" : "ami-414b7271",
"bootstrapScript" : "",
"config" : {
"subnetId" : "subnet-1e1f6f47",
"rootVolumeType" : "gp2",
"instanceNamePrefix" : "director",
"rootVolumeSizeGB" : "50",
"useSpotInstances" : "false",
"securityGroupsIds" : "sg-fe2aca99"
},
"tags" : {
"Name" : "NewPlatform-ClouderaManager"
},
"normalizeInstance" : true
}
},
"capabilities" : {
"operatingSystemType" : "REDHAT_COMPATIBLE",
"operatingSystemVersion" : "REDHAT_COMPATIBLE_6",
"virtualizationType" : "HARDWARE_ASSISTED",
"pythonVersion" : "2.6.6",
"packageManager" : "YUM",
"passwordlessSudoEnabled" : true,
"selinuxEnabled" : true,
"iptablesEnabled" : false
},
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
}
}, {
"ipAddress" : "10.170.1.107",
"properties" : {
"subnetId" : "subnet-1e1f6f47",
"ebsOptimized" : "false",
"virtualizationType" : "hvm",
"imageId" : "ami-414b7271",
"instanceType" : "m4.large",
"keyName" : "NewPlatformRoot",
"publicDnsName" : "",
"sourceDestCheck" : "true",
"availabilityZone" : "us-west-2c",
"privateIpAddress" : "10.170.1.107",
"rootDeviceType" : "ebs",
"launchTime" : "Fri Jan 22 09:59:22 EST 2016",
"instanceId" : "i-729e08a8",
"placementGroupName" : "",
"hypervisor" : "xen",
"spot" : "false",
"vpcId" : "vpc-1b47767e",
"rootDeviceName" : "/dev/sda1",
"privateDnsName" : "ip-10-170-1-107.us-west-2.compute.internal",
"securityGroups" : "launch-wizard-1(sg-fe2aca99)",
"architecture" : "x86_64"
},
"virtualInstance" : {
"id" : "b3ca74d7-4a82-49d2-8dbd-44cac2fad24a",
"template" : {
"name" : "ClouderaManager",
"type" : "m4.large",
"image" : "ami-414b7271",
"bootstrapScript" : "",
"config" : {
"subnetId" : "subnet-1e1f6f47",
"rootVolumeType" : "gp2",
"instanceNamePrefix" : "director",
"rootVolumeSizeGB" : "50",
"useSpotInstances" : "false",
"securityGroupsIds" : "sg-fe2aca99"
},
"tags" : {
"Name" : "NewPlatform-ClouderaManager"
},
"normalizeInstance" : true
}
},
"capabilities" : {
"operatingSystemType" : "REDHAT_COMPATIBLE",
"operatingSystemVersion" : "REDHAT_COMPATIBLE_6",
"virtualizationType" : "HARDWARE_ASSISTED",
"pythonVersion" : "2.6.6",
"packageManager" : "YUM",
"passwordlessSudoEnabled" : true,
"selinuxEnabled" : true,
"iptablesEnabled" : false
},
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
}
}, {
"ipAddress" : "10.170.1.106",
"properties" : {
"subnetId" : "subnet-1e1f6f47",
"ebsOptimized" : "false",
"virtualizationType" : "hvm",
"imageId" : "ami-414b7271",
"instanceType" : "m4.large",
"keyName" : "NewPlatformRoot",
"publicDnsName" : "",
"sourceDestCheck" : "true",
"availabilityZone" : "us-west-2c",
"privateIpAddress" : "10.170.1.106",
"rootDeviceType" : "ebs",
"launchTime" : "Fri Jan 22 09:59:22 EST 2016",
"instanceId" : "i-739e08a9",
"placementGroupName" : "",
"hypervisor" : "xen",
"spot" : "false",
"vpcId" : "vpc-1b47767e",
"rootDeviceName" : "/dev/sda1",
"privateDnsName" : "ip-10-170-1-106.us-west-2.compute.internal",
"securityGroups" : "launch-wizard-1(sg-fe2aca99)",
"architecture" : "x86_64"
},
"virtualInstance" : {
"id" : "d097a064-82a1-4ef8-b108-fd64edb0450d",
"template" : {
"name" : "ClouderaManager",
"type" : "m4.large",
"image" : "ami-414b7271",
"bootstrapScript" : "",
"config" : {
"subnetId" : "subnet-1e1f6f47",
"rootVolumeType" : "gp2",
"instanceNamePrefix" : "director",
"rootVolumeSizeGB" : "50",
"useSpotInstances" : "false",
"securityGroupsIds" : "sg-fe2aca99"
},
"tags" : {
"Name" : "NewPlatform-ClouderaManager"
},
"normalizeInstance" : true
}
},
"capabilities" : {
"operatingSystemType" : "REDHAT_COMPATIBLE",
"operatingSystemVersion" : "REDHAT_COMPATIBLE_6",
"virtualizationType" : "HARDWARE_ASSISTED",
"pythonVersion" : "2.6.6",
"packageManager" : "YUM",
"passwordlessSudoEnabled" : true,
"selinuxEnabled" : true,
"iptablesEnabled" : false
},
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
}
}, {
"ipAddress" : "10.170.0.142",
"properties" : {
"subnetId" : "subnet-1e1f6f47",
"ebsOptimized" : "false",
"virtualizationType" : "hvm",
"imageId" : "ami-414b7271",
"instanceType" : "m4.large",
"keyName" : "NewPlatformRoot",
"publicDnsName" : "",
"sourceDestCheck" : "true",
"availabilityZone" : "us-west-2c",
"privateIpAddress" : "10.170.0.142",
"rootDeviceType" : "ebs",
"launchTime" : "Fri Jan 22 09:59:22 EST 2016",
"instanceId" : "i-f8910722",
"placementGroupName" : "",
"hypervisor" : "xen",
"spot" : "false",
"vpcId" : "vpc-1b47767e",
"rootDeviceName" : "/dev/sda1",
"privateDnsName" : "ip-10-170-0-142.us-west-2.compute.internal",
"securityGroups" : "launch-wizard-1(sg-fe2aca99)",
"architecture" : "x86_64"
},
"virtualInstance" : {
"id" : "acaa5119-b13f-4051-8841-7567e15e30e5",
"template" : {
"name" : "ClouderaManager",
"type" : "m4.large",
"image" : "ami-414b7271",
"bootstrapScript" : "",
"config" : {
"subnetId" : "subnet-1e1f6f47",
"rootVolumeType" : "gp2",
"instanceNamePrefix" : "director",
"rootVolumeSizeGB" : "50",
"useSpotInstances" : "false",
"securityGroupsIds" : "sg-fe2aca99"
},
"tags" : {
"Name" : "NewPlatform-ClouderaManager"
},
"normalizeInstance" : true
}
},
"capabilities" : {
"operatingSystemType" : "REDHAT_COMPATIBLE",
"operatingSystemVersion" : "REDHAT_COMPATIBLE_6",
"virtualizationType" : "HARDWARE_ASSISTED",
"pythonVersion" : "2.6.6",
"packageManager" : "YUM",
"passwordlessSudoEnabled" : true,
"selinuxEnabled" : true,
"iptablesEnabled" : false
},
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
}
} ],
"services" : [ {
"serviceName" : "ZOOKEEPER",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-ZOOKEEPER-OwinYWHV",
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-ZOOKEEPER-OwinYWHV/ins...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
}, {
"serviceName" : "SQOOP",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-SQOOP-UrPslgqE",
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-SQOOP-UrPslgqE/instanc...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
}, {
"serviceName" : "SPARK_ON_YARN",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-SPARK_ON_YARN-AJfBtbso...
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-SPARK_ON_YARN-AJfBtbso...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
}, {
"serviceName" : "HIVE",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HIVE-QAFjCOQn",
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HIVE-QAFjCOQn/instance...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
}, {
"serviceName" : "OOZIE",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-OOZIE-wMmgwoTc",
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-OOZIE-wMmgwoTc/instanc...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
}, {
"serviceName" : "HUE",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HUE-UBSUTqcp",
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HUE-UBSUTqcp/instances...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
}, {
"serviceName" : "YARN",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-YARN-Cahmhgyy",
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-YARN-Cahmhgyy/instance...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
}, {
"serviceName" : "HDFS",
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HDFS-Fxvxaehf",
"roleInstancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HDFS-Fxvxaehf/instance...
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453733448326
},
"healthChecks" : [ ]
} ],
"url" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/clusterRedirect/Cluster01",
"instancesUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/clusterRedirect/Cluster01/hosts",
"health" : {
"status" : "NOT_AVAILABLE",
"lastReported" : 1453821883390
}
}

avatar
Contributor

Here is the same information going directly through the Cloudera Manager API (instead of director). 

 

[ec2-user@ip-10-170-0-201 ~]$ curl --user admin:<redacted> http://10.170.2.146:7180/api/v7/clusters/Cluster01/services
{
"items" : [ {
"name" : "CD-HIVE-QAFjCOQn",
"type" : "HIVE",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HIVE-QAFjCOQn",
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "HIVE_HIVEMETASTORES_HEALTHY",
"summary" : "GOOD"
}, {
"name" : "HIVE_HIVESERVER2S_HEALTHY",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "HIVE-1"
}, {
"name" : "CD-HUE-UBSUTqcp",
"type" : "HUE",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HUE-UBSUTqcp",
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "HUE_HUE_SERVERS_HEALTHY",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "HUE-1"
}, {
"name" : "CD-OOZIE-wMmgwoTc",
"type" : "OOZIE",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-OOZIE-wMmgwoTc",
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "OOZIE_OOZIE_SERVERS_HEALTHY",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "OOZIE-1"
}, {
"name" : "CD-SPARK_ON_YARN-AJfBtbso",
"type" : "SPARK_ON_YARN",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-SPARK_ON_YARN-AJfBtbso...
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "SPARK_ON_YARN-1"
}, {
"name" : "CD-SQOOP-UrPslgqE",
"type" : "SQOOP",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-SQOOP-UrPslgqE",
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "SQOOP_SQOOP_SERVER_HEALTH",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "SQOOP-1"
}, {
"name" : "CD-YARN-Cahmhgyy",
"type" : "YARN",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-YARN-Cahmhgyy",
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "YARN_JOBHISTORY_HEALTH",
"summary" : "GOOD"
}, {
"name" : "YARN_NODE_MANAGERS_HEALTHY",
"summary" : "GOOD"
}, {
"name" : "YARN_RESOURCEMANAGERS_HEALTH",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "YARN-1"
}, {
"name" : "CD-HDFS-Fxvxaehf",
"type" : "HDFS",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-HDFS-Fxvxaehf",
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "HDFS_BLOCKS_WITH_CORRUPT_REPLICAS",
"summary" : "GOOD"
}, {
"name" : "HDFS_CANARY_HEALTH",
"summary" : "GOOD"
}, {
"name" : "HDFS_DATA_NODES_HEALTHY",
"summary" : "GOOD"
}, {
"name" : "HDFS_FREE_SPACE_REMAINING",
"summary" : "GOOD"
}, {
"name" : "HDFS_HA_NAMENODE_HEALTH",
"summary" : "GOOD"
}, {
"name" : "HDFS_MISSING_BLOCKS",
"summary" : "GOOD"
}, {
"name" : "HDFS_UNDER_REPLICATED_BLOCKS",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "HDFS-1"
}, {
"name" : "CD-ZOOKEEPER-OwinYWHV",
"type" : "ZOOKEEPER",
"clusterRef" : {
"clusterName" : "Cluster01"
},
"serviceUrl" : "http://ip-10-170-2-146.us-west-2.compute.internal:7180/cmf/serviceRedirect/CD-ZOOKEEPER-OwinYWHV",
"serviceState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "ZOOKEEPER_CANARY_HEALTH",
"summary" : "GOOD"
}, {
"name" : "ZOOKEEPER_SERVERS_HEALTHY",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "FRESH",
"clientConfigStalenessStatus" : "FRESH",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"displayName" : "ZOOKEEPER-1"
} ]
}[ec2-user@ip-10-170-0-201 ~]$

avatar
Rising Star

Hi,

 

Thanks for the information you provided. Since you have changed the Cloudera Manager credentials, it is possible to update Director to use the same, which should restore access, and have the health show up again.

 

On your Director, navigate to the status page for the deployment, i.e. for the CM. On the right there is a drop down menu, click on "Update Cloudera Manager Credentials". This should pop up a form where you can save the new credentials for Director. After a bit the health for the cluster will show up...

 

Hope this helps,

Jayita

avatar
Contributor

This worked! Thank you !!