Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

HIVE MAJOR Compaction fails for one specific partition

New Contributor

Hello Experts,


We are using HDP 3.1.5 and since short time we have one specific table with several partitions created in it, where the compaction for one of these partitions fails repeatly. I'm not able to find the exact error why it is failing.


from yarn I see the following error in the application log: Setting job diagnostics to Task failed task_1657267798389_0085_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0


and in Metastore Log I only find this:
ERROR [thread-71]: compactor.Worker ( - Caught exception while trying to compact
stWriteId:0. Marking failed to avoid repeated failures, Major compactor job failed for thread-71-compactor-<dbname>.<tablename>.<partitionname>! Hadoop JobId: job_1657267798389_0085
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.launchCompactionJob(
at org.apache.hadoop.hive.ql.txn.compactor.Worker$
at Method)


I'm unable to find any more information on what going wrong here.


Best Regards


Super Collaborator

@mareb  Can you please attach application logs for application_1657267798389_0085

New Contributor

I can provide an application log from the latest compaction run. maybe I'm blind, but I don't find a method to upload a text file here.
and posting the entire logfile content exceeds the limit of 10k characters.

Community Manager

@mareb you could use a code/text sharing service to store the code and share the link here

Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.