https://community.cloudera.com/t5/Archives-of-Support-Questions/Compactor-MR-task-is-failing-with-file-not-found-error/m-p/162493#M45173 <P>Discovered the problem was the hive user account did not have write permissions to the directory containing the hive database. After fixing the permissions the compactor ran succesfully.</P> Sat, 05 Nov 2016 03:35:49 GMT bhopp 2016-11-05T03:35:49Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Compactor-MR-task-is-failing-with-file-not-found-error/m-p/162492#M45172 <P>In my lab environment I have a transactional hive table setup that is receiving data from NiFi. Everything works except the number of delta directories is blowing up because the compactor is not successfully running. It appears to be looking for a _tmp file that does not exist. This is the error that I am getting:</P><PRE>No of maps and reduces are 0 job_1478131263487_0009 Job commit failed: java.io.FileNotFoundException: File hdfs://node2.localdomain:8020/apps/hive/warehouse/stock.db/hdp/_tmp_80e50691-c9fe-485a-8e1e-ba20331c7d97 does not exist. at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904) at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113) at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:966) at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:962) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:962) at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorOutputCommitter.commitJob(CompactorMR.java:776) at org.apache.hadoop.mapred.OutputCommitter.commitJob(OutputCommitter.java:291) at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:285) at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) </PRE> Thu, 03 Nov 2016 08:44:17 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Compactor-MR-task-is-failing-with-file-not-found-error/m-p/162492#M45172 bhopp 2016-11-03T08:44:17Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Compactor-MR-task-is-failing-with-file-not-found-error/m-p/162493#M45173 <P>Discovered the problem was the hive user account did not have write permissions to the directory containing the hive database. After fixing the permissions the compactor ran succesfully.</P> Sat, 05 Nov 2016 03:35:49 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Compactor-MR-task-is-failing-with-file-not-found-error/m-p/162493#M45173 bhopp 2016-11-05T03:35:49Z