Force closing a HDFS file still open (because uncorrectly copied)

[#1]

Hello,

We have a file which we can't backup by distcp due to a mismatch lenght. After a fsck on this file I can see it is still open, but after speaking with file owner it was not. Searching on application logs show us this message:

2017-10-31 18:28:13.466 INFO  [pr-8243-exec-12][o.a.hadoop.hdfs.DFSClient] Unable to close file because dfsclient  was unable to contact the HDFS servers. clientRunning false hdfsTimeout -1
2017-10-31 18:28:13.468 ERROR [pr-8243-exec-12][r.c.a.LogExceptionHandler] error while processing request - uri: /xxx/yyy- query string: [zzzzzzzzzzzzz] - exception: java.io.IOException: Unable to close file because dfsclient  was unable to contact the HDFS servers. clientRunning false hdfsTimeout -1
com.aaaa.bbb.ccc.ddd.exception.AmethystRuntimeException: java.io.IOException: Unable to close file because dfsclient  was unable to contact the HDFS servers. clientRunning false hdfsTimeout -1

We restart HDFS service at this hour, so we think it's the main reason of this problem.

Copying the file by a "hdfs dfs -cp" create a new file correctly closed, so we probably can replace the unclosed file. Nevertheless I want to know if there is another simple method to directly close a file still opened from a HDFS point of vue?

Thanks for your help

[#2 · Accepted solution]

hdfs debug recoverLease command should be used to close the file. The complete syntax is like below:

hdfs debug recoverLease -path <path-of-the-file>[-retries <retry-times>]

[#3]

Thank you @Xiaoyu Yao it works!