Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎06-16-2015

Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

I keep getting the below error 

 

I have seen a few postings related to this but no solution for what I should be looking for

 

Logs not available for attempt_1434490646451_0004_m_000000_0. Aggregation may not be complete, Check back later or try the nodemanager at quickstart.cloudera:8041
Cloudera Employee
Posts: 306
Registered: ‎01-16-2014

Re: Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

Check in CM if log aggregation is turned on under the yarn service.

Go to the yarn service, configuration, type aggregation in the search box.

 

All the settings you need to check are there (3 in total).

 

Wilfred

New Contributor
Posts: 2
Registered: ‎05-08-2014

Re: Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

I ran into this problem myself.  What I found with the 5.4 Quickstart VM (cloudera-quickstart-vm-5.4.0-0-virtualbox.ovf) was that log aggregation worked fine initially but when I turned on Cloudera Manager it broke.

 

What I had to do to fix it was:

  • revert the "yarn.nodemanager.remote-app-log-dir" configuration setting (Remote App Log Directory in NodeManager Base Group) to the default of /tmp/logs -- for some reason this is using a non-default value and I don't think the JHS would be seeing that configuration so it would look in the wrong place
  • run the following commands to set up the /tmp/logs directory with the correct user/group permissions - without this you get an error trying to look at the logs in the JHS web UI.
sudo -u hdfs hadoop fs -mkdir -p /tmp/logs
sudo -u hdfs hadoop fs -chown mapred /tmp/logs
sudo -u hdfs hadoop fs -chgrp hadoop /tmp/logs
sudo -u hdfs hadoop fs -chmod 1777 /tmp/logs
 
  • restart the NodeManager

 

After this log aggregation was working properly and I could view the logs with "yarn logs -applicationId <appId>" or view the Job History Server web UI.

Cloudera Employee
Posts: 435
Registered: ‎07-12-2013

Re: Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

Thanks for sharing your solution, Ben. I'll get that looked at in the next
release of the VM.