Member since
09-27-2021
3
Posts
0
Kudos Received
0
Solutions
09-27-2021
06:25 PM
compacte major got the same error
... View more
09-27-2021
04:10 PM
yes, it is a MANAGED table, there is no base dir only delta dir. i will try MAJOR compaction. thanks.
... View more
09-27-2021
07:58 AM
No of maps and reduces are 0 job_1632722322222_0037 Job commit failed: java.io.FileNotFoundException: File hdfs://xxx/user/hive/warehouse/xxx.db/table/dt=2021-09-25/hr=00/_tmp_d5b133c1-68a1-4e98-af45-308ffbdc9f42 does not exist. at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:986) at org.apache.hadoop.hdfs.DistributedFileSystem.access$1000(DistributedFileSystem.java:122) at org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1046) at org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1043) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:1053) at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorOutputCommitter.commitJob(CompactorMR.java:781) at org.apache.hadoop.mapred.OutputCommitter.commitJob(OutputCommitter.java:291) at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:286) at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:238) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) I had configed the MapReduce excutor root as hadoop proxyuser like this: hadoop.proxyuser.root.hosts=* hadoop.proxyuser.root.groups=* I changed the file owner to root and chmod 777 of the file but still got error "_tmp_d5b133c1-68a1-4e98-af45-308ffbdc9f44 does not exist." please help me, thanks.
... View more
Labels:
- Labels:
-
Apache Hive
-
HDFS