Unable to access logs CDH 5.4.x Quick start VM both from CLI and Job History Server

[kvajjala]

I keep getting the below error 

I have seen a few postings related to this but no solution for what I should be looking for

Logs not available for attempt_1434490646451_0004_m_000000_0. Aggregation may not be complete, Check back later or try the nodemanager at quickstart.cloudera:8041

[Wilfred]

Check in CM if log aggregation is turned on under the yarn service.

Go to the yarn service, configuration, type aggregation in the search box.

All the settings you need to check are there (3 in total).

Wilfred

[benroling]

I ran into this problem myself.  What I found with the 5.4 Quickstart VM (cloudera-quickstart-vm-5.4.0-0-virtualbox.ovf) was that log aggregation worked fine initially but when I turned on Cloudera Manager it broke.

What I had to do to fix it was:

• revert the "yarn.nodemanager.remote-app-log-dir" configuration setting (Remote App Log Directory in NodeManager Base Group) to the default of /tmp/logs -- for some reason this is using a non-default value and I don't think the JHS would be seeing that configuration so it would look in the wrong place
• run the following commands to set up the /tmp/logs directory with the correct user/group permissions - without this you get an error trying to look at the logs in the JHS web UI.

sudo -u hdfs hadoop fs -mkdir -p /tmp/logs

sudo -u hdfs hadoop fs -chown mapred /tmp/logs
sudo -u hdfs hadoop fs -chgrp hadoop /tmp/logs

sudo -u hdfs hadoop fs -chmod 1777 /tmp/logs

• restart the NodeManager

After this log aggregation was working properly and I could view the logs with "yarn logs -applicationId <appId>" or view the Job History Server web UI.

[Sean]

Thanks for sharing your solution, Ben. I'll get that looked at in the next
release of the VM.