question Re: Hive Msck Repair Table Hangs in Support Questions

Hive Msck Repair Table Hangs

hanumanth — Wed, 28 Jun 2023 02:35:18 GMT

Hi Team,

When we are running Msck Repair Table command it is hanging , can anyone please help me on this issue??

Regards,

Hanu

Re: Hive Msck Repair Table Hangs

udeshmukh — Wed, 28 Jun 2023 05:24:16 GMT

Hi Hanu,

if the MSCK is hanging for a particular table, then there must be some problem on that table,

What do you see in HS2 or HMS logs when the command is hung ? Are you seeing any error in logs or ? Can you share the thread which is running MSCK ? Check what that thread is doing, it will give an idea where the problem is coming from.

Re: Hive Msck Repair Table Hangs

VidyaSargur — Mon, 10 Jul 2023 12:01:20 GMT

@hanumanth, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

Re: Hive Msck Repair Table Hangs

hanumanth — Thu, 13 Jul 2023 12:19:26 GMT

Hi @udeshmukh ,

I asked user to check from table level, as there was 36k partitions was there on that table and they are dropping and re-create table and then running msck.

Regards,

Hanu

Re: Hive Msck Repair Table Hangs

udeshmukh — Fri, 14 Jul 2023 03:48:57 GMT

@hanumanth If its an external table, dropping it will not delete the data from HDFS and when you recreate it, the MSCK again has to go through all the 36k partitions and scan them, this may overload the HMS for time being and can cause slowness.

Re: Hive Msck Repair Table Hangs

hanumanth — Fri, 14 Jul 2023 04:38:03 GMT

@udeshmukh it is external table only, same i explained to end user. I asked them to check at table level why 36k partitions are listed??

From our admin side any solution we can recommend for the users??

Re: Hive Msck Repair Table Hangs

udeshmukh — Thu, 20 Jul 2023 05:19:54 GMT

Hi @hanumanth

The MSCK will run slow for the first time as it has to go through all the partitions and update HMS.

You can recommend users to maintain around 10k partitions to avoid slowness/hung in HMS

Re: Hive Msck Repair Table Hangs

VidyaSargur — Tue, 25 Jul 2023 05:27:24 GMT

@hanumanth, did the responses from@udeshmukh assist you in resolving your concerns? If so, could you kindly mark the most suitable response as the solution? This will be beneficial for other members who might come across a similar issue.