Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

New Contributor

I keep getting the below error 

 

I have seen a few postings related to this but no solution for what I should be looking for

 

Logs not available for attempt_1434490646451_0004_m_000000_0. Aggregation may not be complete, Check back later or try the nodemanager at quickstart.cloudera:8041
3 REPLIES 3

Re: Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

Super Collaborator

Check in CM if log aggregation is turned on under the yarn service.

Go to the yarn service, configuration, type aggregation in the search box.

 

All the settings you need to check are there (3 in total).

 

Wilfred

Highlighted

Re: Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

New Contributor

I ran into this problem myself.  What I found with the 5.4 Quickstart VM (cloudera-quickstart-vm-5.4.0-0-virtualbox.ovf) was that log aggregation worked fine initially but when I turned on Cloudera Manager it broke.

 

What I had to do to fix it was:

  • revert the "yarn.nodemanager.remote-app-log-dir" configuration setting (Remote App Log Directory in NodeManager Base Group) to the default of /tmp/logs -- for some reason this is using a non-default value and I don't think the JHS would be seeing that configuration so it would look in the wrong place
  • run the following commands to set up the /tmp/logs directory with the correct user/group permissions - without this you get an error trying to look at the logs in the JHS web UI.
sudo -u hdfs hadoop fs -mkdir -p /tmp/logs
sudo -u hdfs hadoop fs -chown mapred /tmp/logs
sudo -u hdfs hadoop fs -chgrp hadoop /tmp/logs
sudo -u hdfs hadoop fs -chmod 1777 /tmp/logs
 
  • restart the NodeManager

 

After this log aggregation was working properly and I could view the logs with "yarn logs -applicationId <appId>" or view the Job History Server web UI.

Re: Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

Master Collaborator
Thanks for sharing your solution, Ben. I'll get that looked at in the next
release of the VM.