Member since
12-08-2016
2
Posts
2
Kudos Received
0
Solutions
12-08-2016
01:51 AM
Yeah, I tried it and it worked even though still didn't try it with more than 170 directories. However, I really need to use MSCK REPAIR TABLE instead of ALTER TABLE to add thousands of partitions at a time.
... View more
12-08-2016
01:26 AM
2 Kudos
Hello, I installed HDP-2.5.3 recently and have an issue with MSCK REPAIR TABLE. I tried to migrate data from another cluster and created an external hive table on it. The command was successfully done with until 170 data directories and it was even very quick like 3 seconds. However, when I tried it with 190 or more data directories, it was hanging somewhere until I killed it in a few hours. I looked at hivemetastore.log and found it didn't proceed any more after having called 'get_partitions'. I tested it with data with much more directories like thousands in another cluster where HDP-2.4.X is installed and it worked without any problem. And 'get_partitions_with_auth' was called instead of 'get_partitions'. I compared their configs one by one in ambari but don't see any difference. Does anyone have any ideas?
... View more
Labels:
- Labels:
-
Apache Hive