Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can I delete multiple files from hdfs?

Solved Go to solution

Can I delete multiple files from hdfs?

New Contributor

Currently we are seeing our HDFS DFS directory is getting filled up and we have to remove the data at faster rate.

We currently have 12 datanodes and 4 masternodes 1 edgenode. Can I delete the files from HDFS from masternodes and edgenodes at once? I have created a script on edgenode which deletes the HDFS files but speed is really slow.

How can I delete multiple files at a time ? Can I place that script on multiple server and delete the files?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Can I delete multiple files from hdfs?

New Contributor

Hello, Madhura Mhatre!

You can try to use tHDFSList to iterate each file that you want to delete.

For example:

tHDFSList--iterate--tHDFSDelete

There is a global variable that stores the current file path on tHDFSList:

(String)globalMap.get("tHDFSList_1_CURRENT_FILEPATH")

Set the file path of tHDFSdDelete with this variable.

Also please check this article on StackOverflow

This video

I hope it helps!

1 REPLY 1

Re: Can I delete multiple files from hdfs?

New Contributor

Hello, Madhura Mhatre!

You can try to use tHDFSList to iterate each file that you want to delete.

For example:

tHDFSList--iterate--tHDFSDelete

There is a global variable that stores the current file path on tHDFSList:

(String)globalMap.get("tHDFSList_1_CURRENT_FILEPATH")

Set the file path of tHDFSdDelete with this variable.

Also please check this article on StackOverflow

This video

I hope it helps!