Since the partition related information not mentioned in the write statement staging directory created in the table directory instead of partition directory.  ... View more

To determine the cause of the failure, it is recommended to review the HMS logs within the specified time frame as the exception stack-trace does not provide sufficient information. ... View more

make sure below is enabled at cluster level.  hive.acid.direct.insert.enabled Also use below format to insert into partitioned tables.  Static partition   df.write.format(HIVE_WAREHOUSE_CONNECTOR).mode("append").option("partition", "c1='val1',c2='val2'").option("table", "t1").save();   Dynamic partition   df.write.format(HIVE_WAREHOUSE_CONNECTOR).mode("append").option("partition", "c1,c2").option("table", "t1").save();   ... View more

It looks similar to the KB Please follow the instructions in the KB.  ... View more

Please verify if there are any long-running transactions on the cluster and, if found, consider aborting them using the "abort transactions" command, if it is safe to do so. You can use the "show transactions" command in Beeline to validate the long-running transactions. Another alternative is to use the following backend DB query . SELECT * FROM "TXNS" WHERE "TXN_ID" = ( SELECT min(res.id) FROM ( SELECT "NTXN_NEXT" AS id FROM "NEXT_TXN_ID" UNION ALL SELECT "MHL_TXNID" FROM "MIN_HISTORY_LEVEL" WHERE "MHL_TXNID" = ( SELECT min("MHL_MIN_OPEN_TXNID") FROM "MIN_HISTORY_LEVEL" ) ) res) Note: This query is for postgres DB,  modify it depending upon the backend DB in which you're using.  ... View more

From the Error Stack-trace , it does looks like problem is , connection closed exception between Tez AM application and the HBase server (host12.com). This connection closure led to a java.net.SocketTimeoutException. In simpler terms, tez AM tried to communicate with HBase but the connection timed out after 60 seconds (callTimeout) because it didn't receive a response within that time frame. This issue occurred during the initialization of a vertex named "vertex_1723487266861_0004_2_00" within the Tez application. Possible Reasons The HBase server might be overloaded, experiencing internal issues, or even down entirely. Hive might have an incorrect HBase server address or port configured.   ... View more

Can you please share the stacktrace from HiveServer2 logs  and spark-submit command used ... View more

Could you please share the cluster version and spark-submit command .  What's the HWC execution mode? Can you please share the complete StackTrace? As the issue is with respect to MoveTask , HIVE-24163  can be a problem.  ... View more

Is the query failed in compilation stage / execution stage? Could you please share the complete stack-trace of the query failure? ... View more

Could you please provide a brief explanation of the issue? Is there a query stuck in Hive? If so, could you share the query details and specify whether it gets stuck during the compilation phase or the execution phase? Additionally, does the internal pause refer to a JVM pause? If it does, is the pause occurring in HiveServer2 or Hive Metastore? In this scenario, a heap dump would provide better clarity on the context. ... View more