Reply
New Contributor
Posts: 2
Registered: ‎11-08-2017

Impala Catalog Server gets Web Server Status Issue after each restart

Hi,

 

I just installed Cloudera Manager 5.13 and "Parcel" installed the "all services" cluster. I haven't run anything on the cluster yet. But the Impala goes down and give me this:

 

"Bad : The Cloudera Manager Agent got an unexpected response from this role's web server."

 

As details, it says:

 

"The Cloudera Manager Agent got an unexpected response from this role's web server."

and

"The health test result for CATALOGSERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent got an unexpected response from this role's web server."

 

It seems meaning that my catalog server send sth to the agent which was not understandable. So I checked the log of my server and two daemons. (I restarted the service at 9:46. The web server issue happened on 9:48:34)

 

Impala Catalog Server:

 

9:46:38.169 AMINFOcc:110
CatalogService started on port: 26000
9:46:40.110 AMINFOcc:241
Catalog Version: 3 Last Catalog Version: 0
9:47:03.068 AMINFOcc:296
Old log file deleted during log rotation: /var/log/catalogd/catalogd.master.impala.log.INFO.20171108-214603.2421
9:47:03.068 AMINFOcc:296
Old log file deleted during log rotation: /var/log/catalogd/catalogd.master.impala.log.WARNING.20171108-214603.2421
9:47:03.068 AMINFOcc:296
Old log file deleted during log rotation: /var/log/catalogd/catalogd.master.impala.log.ERROR.20171108-214603.2421
9:56:40.166 AMINFOcc:241
Catalog Version: 3 Last Catalog Version: 3
10:06:40.219 AMINFOcc:241
Catalog Version: 3 Last Catalog Version: 3

 

Impala Daemons (these two are pretty same so I just post one of them here):

9:46:33.795 AMINFOcc:113
Impala has started.
9:47:01.823 AMINFOcc:296
Old log file deleted during log rotation: /var/log/impalad/impalad.slave02.impala.log.INFO.20171108-214602.18311
9:47:01.824 AMINFOcc:296
Old log file deleted during log rotation: /var/log/impalad/impalad.slave02.impala.log.WARNING.20171108-214602.18311
9:47:01.824 AMINFOcc:296
Old log file deleted during log rotation: /var/log/impalad/impalad.slave02.impala.log.ERROR.20171108-214602.18311

 

But I found nothing goes wrong, so as stacks logs.

 

Then I tried to restart the service and the whole cluster. But these operations do not work. The issue always appeares soon after my each restart.

 

 

Could anyone tell me what's the problem and how to resolve it? Thanks!

 

Explorer
Posts: 6
Registered: ‎12-28-2017

Re: Impala Catalog Server gets Web Server Status Issue after each restart

Hello !

 

I'm facing the same issue, can you post what you have found if y've already been fixing this issue ?

 

Many thanks

Posts: 509
Topics: 1
Kudos: 116
Solutions: 63
Registered: ‎04-22-2014

Re: Impala Catalog Server gets Web Server Status Issue after each restart

@aispand @skhadrao,

 

The health check that fails is run by the Cloudera Manager Agent.

On the host where the problem occurs, review your agent log file (/var/log/cloudera-scm-agent/cloudera-scm-agent.log by default) and check to see what sort of errors or stack trace is displayed when the agent attempts to connect to the server (Catalog Server in this case).

 

That should give us more information to review.

Highlighted
Contributor
Posts: 52
Registered: ‎01-08-2016

Re: Impala Catalog Server gets Web Server Status Issue after each restart

Hello aisp,

 

This is an issue with your cloudera agent on the impacted host. While you are checking the logs of the agent, I would request you to look at the CM: All hosts -> Status column value.

New Contributor
Posts: 2
Registered: ‎11-08-2017

Re: Impala Catalog Server gets Web Server Status Issue after each restart

Hi there,
Unfortunately I have no solution til now. I have to block Impala at last. Maybe you can check others' replies to get some idea.
Announcements