Skip to main content
clouderaskme
clouderaskme
Silver ⭐️
September 8, 2021
Solved

hdfs has two folder with same names

I have two folders in hdfs with same name under the same directory. I want to get rid of one of them, but how do i do that?

sodonnell
sodonnellAnswer
Gold ⭐️
September 15, 2021

Often this happens as there is a "hidden" character at the end of the file or folder name. For example a line break (\n, \r, etc).

 

If you list the files you can get a clue that is the case as usually the output will look strange with an extra line or something there.

 

You can try running a few commands like the following to see if it matches a file:

hdfs dfs -ls $'/path/to/folder\r'
hdfs dfs -ls $'/path/to/folder\n'
hdfs dfs -ls $'/path/to/folder\r\n'

 

If any of those match, then you can delete the incorrect one with a similar command.

If you get no luck with that, then pipe the ls output into "od -c" and it will show the special characters

hdfs dfs -ls /path/to/folder | od -c

View in thread

3 replies

willx
Gold ⭐️⭐️⭐️
willx
Gold ⭐️⭐️⭐️
September 11, 2021

Hi @clouderaskme 

Creating same folder name in same directory is not allowed. 

Test:

# sudo -u hdfs hdfs dfs -mkdir /folder1
# sudo -u hdfs hdfs dfs -mkdir /folder1/subfolder1
# sudo -u hdfs hdfs dfs -mkdir /folder1/subfolder1
mkdir: `/folder1/subfolder1': File exists

 

So if you see two subfolder under folder1 with same name, it may due to contain special characters in name.

 

Can you log into the terminal and execute hdfs commands to check and also show us the output?

hdfs dfs -ls /folder1 | cat -A

 

Regards,

Will

Translate
VidyaSargur
Community Manager
VidyaSargur
Community Manager
September 15, 2021

@clouderaskme Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  If you are still experiencing the issue, can you provide the information @xxx has requested?

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
sodonnell
Gold ⭐️
sodonnellAnswer
Gold ⭐️
September 15, 2021

Often this happens as there is a "hidden" character at the end of the file or folder name. For example a line break (\n, \r, etc).

 

If you list the files you can get a clue that is the case as usually the output will look strange with an extra line or something there.

 

You can try running a few commands like the following to see if it matches a file:

hdfs dfs -ls $'/path/to/folder\r'
hdfs dfs -ls $'/path/to/folder\n'
hdfs dfs -ls $'/path/to/folder\r\n'

 

If any of those match, then you can delete the incorrect one with a similar command.

If you get no luck with that, then pipe the ls output into "od -c" and it will show the special characters

hdfs dfs -ls /path/to/folder | od -c

Translate
    Terms & ConditionsAccessibility statement