HIVE MAJOR Compaction fails for one specific partition

[#1]

Hello Experts,

We are using HDP 3.1.5 and since short time we have one specific table with several partitions created in it, where the compaction for one of these partitions fails repeatly. I'm not able to find the exact error why it is failing.

from yarn I see the following error in the application log:
org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to Task failed task_1657267798389_0085_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0

and in Metastore Log I only find this:
ERROR [thread-71]: compactor.Worker (Worker.java:run(198)) - Caught exception while trying to compact
id:697908,dbname:standard,tableName:<table-name>,partName:<partitionname>,state:^@,type:MAJOR,properties:null,runAs:null,tooManyAborts:false,highe
stWriteId:0. Marking failed to avoid repeated failures, java.io.IOException: Major compactor job failed for thread-71-compactor-<dbname>.<tablename>.<partitionname>! Hadoop JobId: job_1657267798389_0085
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.launchCompactionJob(CompactorMR.java:596)
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.run(CompactorMR.java:322)
at org.apache.hadoop.hive.ql.txn.compactor.Worker$1.run(Worker.java:182)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:179)

I'm unable to find any more information on what going wrong here.

Best Regards

[#2]

@mareb  Can you please attach application logs for application_1657267798389_0085

[#3]

@asish 
I can provide an application log from the latest compaction run. maybe I'm blind, but I don't find a method to upload a text file here.
and posting the entire logfile content exceeds the limit of 10k characters.

[#4]

@mareb you could use a code/text sharing service to store the code and share the link here