Unable to upload a file with special characters in filename to HDFS

[#1]

I am trying to upload some files having special characters in filename and getting below error.

case:1

======

Command: hdfs dfs -put 900-0314-Slide#2.vsi /user/xyz12345
error: put: `900-0314-Slide': No such file or directory

It might be due to # character.

case: 2

======

Command: hdfs dfs -put 'abc.html?C=S;O=A'  /user/xyz12345

error:  No such file or directory

[#2 · Accepted solution]

Hello @AnuradhaV,

Thanks for raising questions in community. We usually don't suggest to put files with special characters in file name to hdfs. If you have to do this you should replace the special characters with URL encoding.

For example, below characters should be encoded as: 

#  encoded as  %23
?  encoded as  %3F
=  encoded as  %3D
;  encoded as  %3B

A full list of URL encoding characters:

https://www.degraeve.com/reference/urlencoding.php

To help you test it out, I created these files in my local and then put them to hdfs:

# ls
900-0314-Slide#2.vsi  abc.html?C=S;O=A

# hdfs dfs -put 900-0314-Slide%232.vsi /tmp/
# hdfs dfs -put abc.html%3FC%3DS%3BO%3DA /tmp/

# hdfs dfs -ls /tmp
Found 2 items
-rw-r--r--   3 hdfs   supergroup          0 2021-09-03 07:07 /tmp/900-0314-Slide#2.vsi
-rw-r--r--   3 hdfs   supergroup          0 2021-09-03 07:15 /tmp/abc.html?C=S;O=A

Thanks,

Will

If the answer helps, please accept as solution and click thumbs up.

[#3]

@AnuradhaV Below Discussion may be applied in this case as well. 

https://community.cloudera.com/t5/Support-Questions/How-to-delete-a-hdfs-path-with-special-character...

[#4]

Thanks for your suggestion. I tried using double slash but it not worked for me.

[#5]

you should "kinit" with your workload user credentials in CDP.before running any Hadoop commands

[#6 · Accepted solution]

Hello @AnuradhaV,

Thanks for raising questions in community. We usually don't suggest to put files with special characters in file name to hdfs. If you have to do this you should replace the special characters with URL encoding.

For example, below characters should be encoded as: 

#  encoded as  %23
?  encoded as  %3F
=  encoded as  %3D
;  encoded as  %3B

A full list of URL encoding characters:

https://www.degraeve.com/reference/urlencoding.php

To help you test it out, I created these files in my local and then put them to hdfs:

# ls
900-0314-Slide#2.vsi  abc.html?C=S;O=A

# hdfs dfs -put 900-0314-Slide%232.vsi /tmp/
# hdfs dfs -put abc.html%3FC%3DS%3BO%3DA /tmp/

# hdfs dfs -ls /tmp
Found 2 items
-rw-r--r--   3 hdfs   supergroup          0 2021-09-03 07:07 /tmp/900-0314-Slide#2.vsi
-rw-r--r--   3 hdfs   supergroup          0 2021-09-03 07:15 /tmp/abc.html?C=S;O=A

Thanks,

Will

If the answer helps, please accept as solution and click thumbs up.

[#7 · Accepted solution]

@AnuradhaV, has any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

[#8]

Thanks a lot Will, it resolved my issue. Thanks for providing encoding link.

Cheers

Anuradha V