Skip to main content
hanumanth
hanumanth
Gold ⭐️
November 15, 2022
Question

Hive external table MSCK and DROP not working..

Hi Team,

 

We have 1 table which has more than 1 lakh partitions. Support team moved data from HDFS to other path and try to do MSCK but it is not working even drop table also. Getting below errors..

 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

 

While drooping partitions also getting same error after 10min

 

can one help me on this..

 

3 replies

asish
Gold ⭐️⭐️⭐️
asish
Gold ⭐️⭐️⭐️
November 15, 2022

As there are lot many partitions,so DROP command would time out.

 

You must remove the partitions using below command:

 

 You need to drop partition manually https://stackoverflow.com/questions/13815179/how-to-update-drop-a-hive-partition

Then invoke drop table.

 

Drop command times out owing to lot many partitions.

Translate
smruti
Gold ⭐️⭐️⭐️
smruti
Gold ⭐️⭐️⭐️
November 15, 2022

@hanumanth Review the logs to get a better idea on what it is failing on. If we have that many partitions, do check what's the value of hive.msck.repair.batch.size and if it is set to 0 at present, you may set this to a value such as 3000 and retry.

Translate
VidyaSargur
Community Manager
VidyaSargur
Community Manager
November 21, 2022

@hanumanth, Have the replies helped resolve your issue? If so, can you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
Terms & ConditionsAccessibility statement