Reply
Expert Contributor
Posts: 87
Registered: ‎06-16-2014

Re: Cloudera manager admin console Issues

Adam ,

Still Cloudera Management service is in Red when i run it . It was showing
" The host's NTP service did not respond to a request for the clock offset. "

Please help .

Thanks
Bala
Thanks
Bala
Cloudera Employee
Posts: 79
Registered: ‎08-29-2013

Re: Cloudera manager admin console Issues

Bala,

 

Each health check within the Cloudera Manager UI gives context around

- what the specific health check looks for

- what a failing or crossed-threshold indicates

- possible steps to fix or remediate the issue

- a direct link to the location in Monitoring settings to adjust a threshold [where/if applicable].

 

These bodies of text aim to help you as the admin self-solve or self-diagnose the issue reported by a health check. Additionally, the documentation for Cloudera Manager Health Checks provides a full index of every check performed so you can read about each one in detail.

 

The Clock Offset health check you mention in this latest post is discussed in the documentation here.

Expert Contributor
Posts: 87
Registered: ‎06-16-2014

Re: Cloudera manager admin console Issues

Thanks Adam and Smark .
So What would be the reason of my CM referring to 127.0.0.0:7180 instead of my <ipaddress>:7180 ???
Thanks
Bala
Cloudera Employee
Posts: 11
Registered: ‎09-17-2013

Re: Cloudera manager admin console Issues

What happens if you run:

telnet 172.18.42.119 7180

 

Can you run that command, and share the last 25 lines or so of /var/log/cloudera-scm-server/cloudera-scm-server.log

 

 

Thanks,

Adam

Expert Contributor
Posts: 87
Registered: ‎06-16-2014

Re: Cloudera manager admin console Issues

Adam , Today i again see this error

The hostname and canonical name for this host are not consistent when checked from a Java process.

 

 

right now my hostname is inp00958.mycomp.ad , hostname -f is inp00958.mycomp.ad .

this is the Cloudera-scm-server.log

2014-06-27 02:44:57,641  WARN [1793099144@agentServer-0:cmf.AgentProtocolImpl@493] Received Process Heartbeat for unknown (or duplicate) process. Ignoring. This is expected to happen once after old process eviction or process deletion (as happens in restarts). id=310 name=null host=7279318e-d702-4c92-82c0-189f2e67490c/ip00958.mycomp.ad
2014-06-27 02:45:02,609  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1078) is still active on service DbService{id=23, name=oozie}.
2014-06-27 02:45:07,638  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1078) is still active on service DbService{id=23, name=oozie}.
2014-06-27 02:45:12,700  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1078) is still active on service DbService{id=23, name=oozie}.
2014-06-27 02:45:12,737  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1078) is still active on service DbService{id=23, name=oozie}.
2014-06-27 02:45:17,792  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1078) is still active on service DbService{id=23, name=oozie}.
2014-06-27 02:45:19,744  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@611] BringUp command (1079) has finished successfully on service DbService{id=23, name=oozie} role DbRole{id=62, name=oozie-OOZIE_SERVER-914e45d11a129704e093f4704118f842, hostName=ip00958.mycomp.ad}.
2014-06-27 02:45:19,749  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@207] BringUp command (1078) has finished on service DbService{id=23, name=oozie}.
2014-06-27 02:45:19,799  INFO [CommandPusher:service.ServiceHandlerRegistry@749] Executing service command Start SvcCmdArgs{targetRoles=[DbRole{id=68, name=hue-HUE_SERVER-914e45d11a129704e093f4704118f842, hostName=ip00958.mycomp.ad}], args=[]}.  Service: DbService{id=26, name=hue}
2014-06-27 02:45:19,812  INFO [CommandPusher:service.ServiceHandlerRegistry@802] Executing role command Start BasicCmdArgs{args=[]}.  Service: DbService{id=26, name=hue} Role: DbRole{id=68, name=hue-HUE_SERVER-914e45d11a129704e093f4704118f842, hostName=ip00958.mycomp.ad}
2014-06-27 02:45:19,845  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@187] Added BringUp command to service DbService{id=26, name=hue}.
2014-06-27 02:45:19,969  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1080) is still active on service DbService{id=26, name=hue}.
2014-06-27 02:45:20,037  WARN [1793099144@agentServer-0:cmf.AgentProtocolImpl@493] Received Process Heartbeat for unknown (or duplicate) process. Ignoring. This is expected to happen once after old process eviction or process deletion (as happens in restarts). id=311 name=null host=7279318e-d702-4c92-82c0-189f2e67490c/ip00958.mycomp.ad
2014-06-27 02:45:25,011  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1080) is still active on service DbService{id=26, name=hue}.
2014-06-27 02:45:30,044  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1080) is still active on service DbService{id=26, name=hue}.
2014-06-27 02:45:35,066  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1080) is still active on service DbService{id=26, name=hue}.
2014-06-27 02:45:40,117  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@200] BringUp command (1080) is still active on service DbService{id=26, name=hue}.
2014-06-27 02:45:41,803  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@611] BringUp command (1081) has finished successfully on service DbService{id=26, name=hue} role DbRole{id=68, name=hue-HUE_SERVER-914e45d11a129704e093f4704118f842, hostName=ip00958.mycomp.ad}.
2014-06-27 02:45:41,811  INFO [CommandPusher:service.AbstractBringUpBringDownCommands@207] BringUp command (1080) has finished on service DbService{id=26, name=hue}.
2014-06-27 02:49:56,737  INFO [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process heartbeats and 0 host heartbeats.
2014-06-27 02:49:56,757  INFO [StaleEntityEviction:cmf.StaleEntityEvictionThread@238] Found no commands older than 2012-06-26T21:19:56.749Z to reap.
2014-06-27 02:49:56,758  INFO [StaleEntityEviction:node.NodeScannerService@335] Reaped 0 requests.
2014-06-27 02:49:56,758  INFO [StaleEntityEviction:node.NodeConfiguratorService@358] Reaped 0 requests.

 

Thanks

Bala

Thanks
Bala
Expert Contributor
Posts: 87
Registered: ‎06-16-2014

Re: Cloudera manager admin console Issues

My management service and cluster are in Red bad health :(
Thanks
Bala
Expert Contributor
Posts: 87
Registered: ‎06-16-2014

Re: Cloudera manager admin console Issues

Hello ,
This error " The hostname and canonical name for this host are not consistent when checked from a Java process " is coming in the middle . And its gone after sometime . I am confused . I have made all the changes to get rid of this but it keeps coming .

Thanks
Bala
Thanks
Bala
Expert Contributor
Posts: 87
Registered: ‎06-16-2014

Re: Cloudera manager admin console Issues

Hello guys ...

Anyone have a solution for this issue.
Thanks
Bala
Highlighted
Cloudera Employee
Posts: 11
Registered: ‎09-17-2013

Re: Cloudera manager admin console Issues

Hi Bala

 

If this issue is showing up only intermittently, I have to wonder if perhaps the different DNS servers that you are using have different information about the hostname.

 

I saw from your resolv.conf that you have 2 DNS servers listed

 

nameserver 172.18.18.210
nameserver 172.18.129.210

 

Can  you validate that both contain the same information mapping your host IP address to the fully qualified hostname that you are using now?

 

Thanks,

Adam

New Contributor
Posts: 1
Registered: ‎05-17-2016

Re: Cloudera manager admin console Issues

Hi Bala/Adam,

 

Did you guys resolve this error. It seems like the same error is troubling me. I get a DNS Resolution error on 2 of my hosts.

 

My host names are Hadoop01 (IP address 192.168.194.27) and Hadoop02 (IP address 192.168.194.28), where as I actually looked at my resolve.conf file in host Hadoop01 which says 

 

nameserver 8.8.8.8
nameserver 192.168.83.132

 

what am I supposed to do edit in this resolve.conf file..? Am I supposed to include my other host Ip address in here..?

 

Appreciate if someone can help me in resloving this error.. Thanks

Announcements