question Re: What is the significance of /.//* in hadoop classpath? in Archives of Support Questions (Read Only)

What is the significance of /.//* in hadoop classpath?

Ravoz — Fri, 16 Sep 2022 10:33:26 GMT

[cloudera@quickstart ~]$ hadoop classpath
/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*

The above is what I get when I use hadoop classpath. I am wondering what is the meaning of

/usr/lib/hadoop/.//*

and also

/usr/lib/hadoop-hdfs/./

I know that * matches anything and . stands for the current directory. But i can't make any sense of them being used like that together with double slashes (//).

Re: What is the significance of /.//* in hadoop classpath?

Harsh J — Mon, 15 Aug 2016 15:41:43 GMT

There's no real significance to it, its just the way the directory detection logic works today. The double slashes on Linux get interpreted as a single one, and does not induce the behaviour in any way (same within Java's CLASSPATH parser, for which this entry is created)

Here's where the classpath entry is being built: /usr/lib/hadoop/libexec/hadoop-config.sh (you can vi this file on your installation)

Sourcing the variables that are being appended to CLASSPATH ('/*' is being appended to whose values) will reveal further that some of them come from the file /usr/lib/hadoop/libexec/hadoop-layout.sh (you can vi this as well) wherein they are set to './'.

As a result the concatenation operations on these strings produce the logic:

"/usr/lib/hadoop" (base) + "/" (separator) + "./" (specific sub-dir path) + "/*" (wildcard suffix)

The output of such a thing would thereby appear as "/usr/lib/hadoop/.//*".

Similar things apply to the non wildcard question, where we look to just add the directory itself onto the classpath.

Understanding-wise this is no different in evaluation than the reduced form of:

/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/*

Hope this helps in reducing the confusion.

Re: What is the significance of /.//* in hadoop classpath?

Ravoz — Tue, 16 Aug 2016 02:28:23 GMT

Thanks Harsh! Nice explanation!