It looks similar to the KB Please follow the instructions in the KB.  ... View more

Please verify if there are any long-running transactions on the cluster and, if found, consider aborting them using the "abort transactions" command, if it is safe to do so. You can use the "show transactions" command in Beeline to validate the long-running transactions. Another alternative is to use the following backend DB query . SELECT * FROM "TXNS" WHERE "TXN_ID" = ( SELECT min(res.id) FROM ( SELECT "NTXN_NEXT" AS id FROM "NEXT_TXN_ID" UNION ALL SELECT "MHL_TXNID" FROM "MIN_HISTORY_LEVEL" WHERE "MHL_TXNID" = ( SELECT min("MHL_MIN_OPEN_TXNID") FROM "MIN_HISTORY_LEVEL" ) ) res) Note: This query is for postgres DB,  modify it depending upon the backend DB in which you're using.  ... View more

From the Error Stack-trace , it does looks like problem is , connection closed exception between Tez AM application and the HBase server (host12.com). This connection closure led to a java.net.SocketTimeoutException. In simpler terms, tez AM tried to communicate with HBase but the connection timed out after 60 seconds (callTimeout) because it didn't receive a response within that time frame. This issue occurred during the initialization of a vertex named "vertex_1723487266861_0004_2_00" within the Tez application. Possible Reasons The HBase server might be overloaded, experiencing internal issues, or even down entirely. Hive might have an incorrect HBase server address or port configured.   ... View more

Can you please share the stacktrace from HiveServer2 logs  and spark-submit command used ... View more

Could you please share the cluster version and spark-submit command .  What's the HWC execution mode? Can you please share the complete StackTrace? As the issue is with respect to MoveTask , HIVE-24163  can be a problem.  ... View more

Is the query failed in compilation stage / execution stage? Could you please share the complete stack-trace of the query failure? ... View more

Could you please provide a brief explanation of the issue? Is there a query stuck in Hive? If so, could you share the query details and specify whether it gets stuck during the compilation phase or the execution phase? Additionally, does the internal pause refer to a JVM pause? If it does, is the pause occurring in HiveServer2 or Hive Metastore? In this scenario, a heap dump would provide better clarity on the context. ... View more

2.  An alternative is to write a script (e.g., Bash) that interacts with Hive and potentially your desired output format. ... View more

@jayes  Hive itself doesn't offer a built-in command like numfile to directly export table data into a specific number of files. However, you can achieve the same using a couple of approaches: 1. Spark:  Read the hive table using sparkSql  >>> df=spark.sql("select * from sample_table")  if it's managed table use HWC session.  In Apache Spark, you can control the number of partitions in a DataFrame using the repartition or coalesce methods. Using coalesce method set the number of partitions for the dataframe. coalesced_df = df.coalesce(5) Write the data  >>> coalesced_df.write.parquet("/tmp/coalesced_df") Result [hive@node4 ~]$ hdfs dfs -ls -h /tmp/coalesced_df Found 6 items -rw-r--r-- 3 hive supergroup 0 2024-06-07 10:49 /tmp/coalesced_df/_SUCCESS -rw-r--r-- 3 hive supergroup 135.0 M 2024-06-07 10:48 /tmp/coalesced_df/part-00000-2e6bf3b8-53fa-4a6e-957e-83769c72e780-c000.snappy.parquet -rw-r--r-- 3 hive supergroup 200.0 M 2024-06-07 10:48 /tmp/coalesced_df/part-00001-2e6bf3b8-53fa-4a6e-957e-83769c72e780-c000.snappy.parquet -rw-r--r-- 3 hive supergroup 68.9 M 2024-06-07 10:49 /tmp/coalesced_df/part-00002-2e6bf3b8-53fa-4a6e-957e-83769c72e780-c000.snappy.parquet -rw-r--r-- 3 hive supergroup 155.4 M 2024-06-07 10:49 /tmp/coalesced_df/part-00003-2e6bf3b8-53fa-4a6e-957e-83769c72e780-c000.snappy.parquet -rw-r--r-- 3 hive supergroup 132.9 M 2024-06-07 10:49 /tmp/coalesced_df/part-00004-2e6bf3b8-53fa-4a6e-957e-83769c72e780-c000.snappy.parquet [hive@node4 ~]$ ... View more

The export table command automatically creates a distributed copy (distcp) job when the table location contains a large number of files. This improves efficiency for handling massive datasets. The size of the exported file will match the size of the table data. You can adjust the memory allocated to the distcp job mappers and reducers if needed to optimize performance for your specific data size ... View more