Member since
08-15-2019
2
Posts
0
Kudos Received
0
Solutions
08-18-2019
09:32 PM
Hi @khosrucse , Removal of these staging file is part of yarn application execution. But, when the yarn application itself is killed, there is not way to remove these files. The new query on top of the same table also does not have reference to these staging files(so that it can be removed by later runs), as the files are generated by Yarn application which is already killed now. For now, the only option is to manually remove the files. Though, you can refer the below link for a different approach to remove these directories. Please note, this is workaround from an end user, and thus, should be implemented with your environment in mind. https://stackoverflow.com/questions/33844381/hive-overwrite-directory-move-process-as-distcp/35583367#35583367 (for your reference )
... View more