question Re: Is hdfs symbolic link available? in Archives of Support Questions (Read Only)

Is hdfs symbolic link available?

jluo374528065 — Fri, 16 Sep 2022 08:58:38 GMT

Per https://issues.apache.org/jira/browse/HDFS-245, symbolic link is implemented in hdfs for several years, but cdh5 still not able to recognize "hadoop fs -nl" command.

So is symbolic link feature available in hdfs? If so, how to create one?

Re: Is hdfs symbolic link available?

Harsh J — Thu, 08 May 2014 22:17:43 GMT

Apache Hadoop's symlink support is not yet production ready. It has been disabled in functionality at the moment, as work on https://issues.apache.org/jira/browse/HDFS-5298 needs to complete before it can be consumed across the platform.

Re: Is hdfs symbolic link available?

Bogolese — Mon, 13 Jun 2016 18:34:47 GMT

Any update on this? It SURE would be nice to have symlinks in HDFS!!!!

Re: Is hdfs symbolic link available?

michaelthoward — Wed, 10 Aug 2016 14:37:25 GMT

We have been hoping for the reintroduction of HDFS symlinks for several years.

Q: Is there any development at all going on related to HDFS symlinks?

From a distance, it seems like this should relatively easy ... all pathname resolution takes place on the namenode.

Q: Does anyone have any simple/brief insight as to why HDFS symlinks might be technically challenging?

Re: Is hdfs symbolic link available?

jh070784 — Fri, 12 Aug 2016 22:15:13 GMT

I might be missing something here but if symbolic links are not supported in HDFS, what is the significance of the method createSymlink in FileContext class?

https://hadoop.apache.org/docs/r2.6.3/api/org/apache/hadoop/fs/FileContext.html#createSymlink(org.apache.hadoop.fs.Path, org.apache.hadoop.fs.Path, boolean)

Re: Is hdfs symbolic link available?

Harsh J — Fri, 12 Aug 2016 22:22:50 GMT

@jh070784 - Please read my previous response. Support right now is disabled for incompleteness of design/impl., not unavailable. API was added before it was later disabled, and calling it will yield an error as the source reads: https://github.com/cloudera/hadoop-common/blob/cdh5.5.1-release/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java#L1404-L1406

@michaelthoward - I am not aware of any major pick up of redoing symlink but if you're interested in history reading and contributing changes please start at https://issues.apache.org/jira/browse/HADOOP-10019 which is the parent JIRA listing all the problems faced with its design and current implementation.