Failed to CREATE_FILE because the file lease is currently owned by DFSCLient

[#1]

Hi I'm getting this error when running one of my jobs.

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException): Failed to CREATE_FILE /Nameservice/qa/eds/data/extract_db/dl4_alco_als_dashboard/.spark-staging-fcc1bfcd-512c-4a68-a4c7-41c8dae5c21d/ld_dt=20200903/src_stm_cd=ALS/part-00000-fcc1bfcd-512c-4a68-a4c7-41c8dae5c21d.c000.csv for DFSClient_NONMAPREDUCE_-853710210_58 on 10.117.210.1062 because this file lease is currently owned by DFSClient_NONMAPREDUCE_-510940321_58 on 10.117.210.121231

How can I resolve this issue?

[#2]

Hello @Monds you can recover the lease on the file, running below command:

#hdfs debug recoverLease -path <path-of-the-file> [-retries <retry-times>]

This command will ask the NameNode to try to recover the lease for the file (successfully close the file if there are still healthy replicas)

Ref: https://blog.cloudera.com/understanding-hdfs-recovery-processes-part-1/