08-02-2017 12:10 AM
I'm using heterogeneous storage, after changing policy for a folder I'm executing:
hdfs mover -p <path>
It is working correctly but after some random number of iterations it stops with no errors. After checking some of the blocks are moved to proper disks but some are still not moved.
Executing the same command few times allows to move all the blocks to correct disks.
Do any of you know why this happens? Is it normal behaviour?
I want to have it working in one execution or at least know why this is happening.
I tried changing many parameters but still the same. I'm using CDH 5.12.0.
02-20-2018 07:44 AM
this topic came up when I searched for the same issue, it seems to be a bug that was introduced with a change in CDH5.10.0 and above versions due to HDFS-11015.
I am working on a fix under HDFS-13174, is it possible that you experienced the same issue as described there?
02-21-2018 12:05 AM
This is not the same issue. I didn't receive any error, after some iterations it just stopped.
I went through the source code and found out that this is a bug (or a feature).
As a workaroud I managed to write a script that executed mover in a loop until data was