Hive Msck Repair Table Hangs

[#1]

Hi Team,

When we are running Msck Repair Table command it is hanging , can anyone please help me on this issue??

Regards,

Hanu

[#2]

Hi Hanu,

if the MSCK is hanging for a particular table, then there must be some problem on that table, 

What do you see in HS2 or HMS logs when the command is hung ? Are you seeing any error in logs or ? Can you share the thread which is running MSCK ? Check what that thread is doing, it will give an idea where the problem is coming from.

[#3]

Hi @udeshmukh ,

I asked user to check from table level, as there was 36k partitions was there on that table and they are dropping and re-create table and then running msck.

Regards,

Hanu

[#4]

@hanumanth, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  

[#5]

@hanumanth If its an external table, dropping it will not delete the data from HDFS and when you recreate it, the MSCK again has to go through all the 36k partitions and scan them, this may overload the HMS for time being and can cause slowness.

[#6]

@udeshmukh it is external table only, same i explained to end user. I asked them to check at table level why 36k partitions are listed??

From our admin side any solution we can recommend for the users??

[#7]

Hi @hanumanth 

The MSCK will run slow for the first time as it has to go through all the partitions and update HMS.

You can recommend users to maintain around 10k partitions to avoid slowness/hung in HMS

[#8 · Accepted solution]

@hanumanth, did the responses from@udeshmukh assist you in resolving your concerns? If so, could you kindly mark the most suitable response as the solution? This will be beneficial for other members who might come across a similar issue.