Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

Unable to issue query: the Host Monitor is not running

Unable to issue query: the Host Monitor is not running

Hello,

 

I've seen other threads on this issue, but the steps on those threads did not help me solve this issue. Thank you for the help!

 

Specs:

Private cloud

CDH 5.4.2

4 instances

 

Background:

 

The cluster was working fine for ~2 weeks. I noticed last week that the service manager was down, but ignored it as all the services were working fine. Today, I tried working with Kafka and got a java.net.ConnectionRefused exception, then noticed I couldn't start spark-shell as it said it can't connect to node2. 

 

Under "Instances" it says I haven't received a heartbeat from node2 for 13 days, all the other nodes seem to be fine.

 

I can't execute "service cloudera-scm-agent start" on node2, this is the output:

 

/etc/init.d/cloudera-scm-agent: line 123: /var/log/cloudera-scm-agent/cloudera-scm-agent.out: Read-only file system
Starting cloudera-scm-agent: /etc/init.d/cloudera-scm-agent: line 128: /var/run/cloudera-scm-agent.pid: Read-only file system
/etc/init.d/cloudera-scm-agent: line 126: /var/log/cloudera-scm-agent/cloudera-scm-agent.out: Read-only file system
[FAILED]

 

 

I've also just tried restarting the cluster. Stumped on this one :/. Thanks for the help!

2 REPLIES 2

Re: Unable to issue query: the Host Monitor is not running

Explorer

First I'd check to make sure /var/log isn't full. Second, that the permissions on it have not changed.

Highlighted

Re: Unable to issue query: the Host Monitor is not running

Super Guru

Hello

 

"Read-only file system" implies that your disks are not writable.  See if you can touch a file in /var/log and if you get the same error, OS/disk troubleshooting is a good start.

 

- Ben