Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive external table MSCK and DROP not working..

avatar
Contributor

Hi Team,

 

We have 1 table which has more than 1 lakh partitions. Support team moved data from HDFS to other path and try to do MSCK but it is not working even drop table also. Getting below errors..

 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

 

While drooping partitions also getting same error after 10min

 

can one help me on this..

 

3 REPLIES 3

avatar
Guru

As there are lot many partitions,so DROP command would time out.

 

You must remove the partitions using below command:

 

 You need to drop partition manually https://stackoverflow.com/questions/13815179/how-to-update-drop-a-hive-partition

Then invoke drop table.

 

Drop command times out owing to lot many partitions.

avatar
Master Collaborator

@hanumanth Review the logs to get a better idea on what it is failing on. If we have that many partitions, do check what's the value of hive.msck.repair.batch.size and if it is set to 0 at present, you may set this to a value such as 3000 and retry.

avatar
Community Manager

@hanumanth, Have the replies helped resolve your issue? If so, can you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: