Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Some of the services in cloud era manager not listening on their respective port

Highlighted

Some of the services in cloud era manager not listening on their respective port

New Contributor

Hi Guys,

 

My cloud era runs on RHEL 7 box with CDH 6. I might have killed few processes by mistake while trying to do something. When I tried checking netstat of all ports related to cloudera Manager and Cloudera Navigator it looks llike below:

netstat -tulpna | grep LISTEN | grep 7180
tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 11521/java

netstat -tulpna | grep LISTEN | grep 7183

netstat -tulpna | grep LISTEN | grep 7182
tcp 0 0 0.0.0.0:7182 0.0.0.0:* LISTEN 11521/java

netstat -tulpna | grep LISTEN | grep 7432

netstat -tulpna | grep LISTEN | grep 9000
tcp 0 0 10.11.65.99:9000 0.0.0.0:* LISTEN 14313/python2

netstat -tulpna | grep LISTEN | grep 19001
tcp 0 0 127.0.0.1:19001 0.0.0.0:* LISTEN 14124/python2

netstat -tulpna | grep LISTEN | grep 7184
tcp 0 0 0.0.0.0:7184 0.0.0.0:* LISTEN 43921/java

netstat -tulpna | grep LISTEN | grep 7185
tcp 0 0 0.0.0.0:7185 0.0.0.0:* LISTEN 43921/java

netstat -tulpna | grep LISTEN | grep 8084
tcp 0 0 0.0.0.0:8084 0.0.0.0:* LISTEN 43921/java

netstat -tulpna | grep LISTEN | grep 10101
tcp 0 0 0.0.0.0:10101 0.0.0.0:* LISTEN 44129/java

netstat -tulpna | grep LISTEN | grep 8086
tcp 0 0 10.11.65.99:8086 0.0.0.0:* LISTEN 43922/java

netstat -tulpna | grep LISTEN | grep 9997
tcp 0 0 10.11.65.99:9997 0.0.0.0:* LISTEN 43922/java

netstat -tulpna | grep LISTEN | grep 9996
tcp 0 0 0.0.0.0:9996 0.0.0.0:* LISTEN 43922/java

netstat -tulpna | grep LISTEN | grep 8087
tcp 0 0 10.11.65.99:8087 0.0.0.0:* LISTEN 43923/java

netstat -tulpna | grep LISTEN | grep 9999
tcp 0 0 10.11.65.99:9999 0.0.0.0:* LISTEN 43923/java

netstat -tulpna | grep LISTEN | grep 9998
tcp 0 0 0.0.0.0:9998 0.0.0.0:* LISTEN 43923/java

netstat -tulpna | grep LISTEN | grep 8091
tcp 0 0 10.11.65.99:8091 0.0.0.0:* LISTEN 43920/java

netstat -tulpna | grep LISTEN | grep 9995
tcp 0 0 10.11.65.99:9995 0.0.0.0:* LISTEN 43920/java

netstat -tulpna | grep LISTEN | grep 9994
tcp 0 0 0.0.0.0:9994 0.0.0.0:* LISTEN 43920/java

netstat -tulpna | grep LISTEN | grep 5678

netstat -tulpna | grep LISTEN | grep 8083

netstat -tulpna | grep LISTEN | grep 7186

netstat -tulpna | grep LISTEN | grep 7187

netstat -tulpna | grep LISTEN | grep 4867

netstat -tulpna | grep LISTEN | grep 7180
tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 11521/java

netstat -tulpna | grep LISTEN | grep 8020
tcp 0 0 10.11.65.99:8020 0.0.0.0:* LISTEN 25858/java

netstat -tulpna | grep LISTEN | grep 50010

 

As you can see some ports are not listening. What would you suggest me to do? 

Thank you

 

4 REPLIES 4

Re: Some of the services in cloud era manager not listening on their respective port

Community Manager

Hi @Navyaa ,

 

You can refer to this documentation link to refer to the ports used by CM and Navigator:

https://www.cloudera.com/documentation/enterprise/6/6.2/topics/cm_ig_ports_cm.html#cm_cn_ports

 

So from the output you sent, it looks like below services are down:

- Reports Manager

- Navigator Metadata Server

- Navigator Audit Server

 

From Cloudera Manager UI, do you find above three services reporting issues? Have you tried to do a stop and then start of the services? If you have not tried yet, I would recommend you to do that from Cloudera Manager UI.

 

Thanks,

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Re: Some of the services in cloud era manager not listening on their respective port

New Contributor

Hi @lwang ,

 

Thank you for the reply. I did refer to the ports from the link you shared before.

 

Bad health reported for Activity Monitor Health, Service Monitor Health, Host Monitor Health, Event Server Health, Alert Publisher Health. I have stopped and started complete cluster, restarted cluster, stopped and started Cloud manager service, restarted Cloud manager service, stopped and started each service seperately. But nothing has really helped. After each service is started it again says : 

1. HBase - Bad : This role's process exited. This role is supposed to be started.

2. HBase - Bad : Master summary: entrada2.iedr.ie (Availability: Unknown, Health: Bad). This health test is bad because the Service Monitor did not find an active Master.

3. Hue server - The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping, agent parcel directory.

4. Hue , Oozie, Impala, Key-Value Store Indexer,  Solr, Zoo Keeper - Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.

5. Spark - Bad : This role's process exited. This role is supposed to be started.

6. YARN - Bad : ResourceManager summary: entrada2.iedr.ie (Availability: Active, Health: Bad). This health test reflects the health of the active ResourceManager.

 

On Hue editor when running any query it says - Invalid method name: 'ExecuteStatement' (code THRIFTAPPLICATION): None

 

And also Clock offset is throwing error: The host's NTP service could not be located or did not respond to a request for the clock offset. But I am using chronyc instead of NTP.

 

 

Thank you.

 

Re: Some of the services in cloud era manager not listening on their respective port

Expert Contributor

If you switched from using ntp to using chrony service then please restart the CM agent on all cluster nodes. This should help with the clock offset related issues.

 

For roles with health alert This role's process exited please review the stdout, stderr and role logs to find out details why the startup failed, those are found in /var/run/cloudera-scm-agent/process directory, lookup e.g. the recent directories for Spark and review the stdout/stderr logs for error messages.

Re: Some of the services in cloud era manager not listening on their respective port

New Contributor

Hi @gzigldrum ,

 

Thank you for the reply. 

 

I did have a look at stderr.log to see errors such as  - java.io.IOException: No space left on device.

Will address those errors.

 

Can you please let me know where do I have to look for switching from using ntp to using chrony service? 

 

Thanks a Mil.