Member since
07-06-2014
2
Posts
1
Kudos Received
0
Solutions
07-09-2014
04:20 AM
1 Kudo
Hi Romain, Thanks for the reply. I did a sudo -u hdfs hadoop fs -chown -R mapred:mapred /tmp/logs When I ran jobs as 'Cloudera', no new logs were written to the above directory. In Hue, I got a message that no logs were available and aggregation may not have been complete. Then I reverted as follows: sudo -u hdfs hadoop fs -chown -R cloudera:supergroup /tmp/logs Then, when the jobs ran, log directories were created in /tmp/logs/cloudera/logs. However, in Hue, I get the original error message. The new directories were created as below: drwxrwx--- - cloudera supergroup 0 2014-07-09 04:03 /tmp/logs/cloudera/logs/application_1404900446410_0005 -rw-r----- 1 cloudera supergroup 3755405 2014-07-09 04:03 /tmp/logs/cloudera/logs/application_1404900446410_0005/localhost.localdomain_8041 The permissions are not correct. In the files in the tmp folder, I can see the log statements from my reduce program. But it is not available in Hue. What should I do? Thanks in advance!
... View more
07-06-2014
08:34 AM
Hello, I am using CDH 5.0 Quickstart VM for VirtualBox. I run a mapreduce job and it completes successfully. The reduce code has several System.out.println statements which should be logged in the job log. But when I open logs in Hue>Jobbrowser, I get the error message: Error getting logs for job_1404657916663_0001 When I see the Hue logs in /var/log/hadoop-mapreduce/hadoop-cmf-yarn-JOBHISTORY-localhost.localdomain.log.out, I find exceptions like the one below: 2014-07-06 08:18:20,767 ERROR org.apache.hadoop.yarn.webapp.View: Error getting logs for job_1404659730687_0001 org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=EXECUTE, inode="/tmp/logs/cloudera/logs":cloudera:supergroup:drwxrwx--- at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:265) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:251) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:205) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:168) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5461) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5443) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPathAccess(FSNamesystem.java:5405) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1680) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1632) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1612) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1586) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:482) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:322) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1986) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1982) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1980) So, I do: hdfs dfs -chmod -R -777 /tmp/logs/cloudera drwxrwxrwx - cloudera supergroup 0 2014-07-06 08:23 /tmp/logs/cloudera/logs Then I run the napreduce job again. I still get the same error in job browser. The permissions to the log directory have reverted. drwxrwx--- - cloudera supergroup 0 2014-07-06 08:23 /tmp/logs/cloudera/logs What should be done to get the logs for the job? Thanks in advance. Lohith.
... View more