Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

What is the significance of /.//* in hadoop classpath?

avatar
New Contributor

[cloudera@quickstart ~]$ hadoop classpath
/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*

 

 

The above is what I get when I use hadoop classpath. I am wondering what is the meaning of

/usr/lib/hadoop/.//*

and also

/usr/lib/hadoop-hdfs/./

 

I know that * matches anything and . stands for the current directory. But i can't make any sense of them being used like that together with double slashes (//).

1 ACCEPTED SOLUTION

avatar
Mentor
There's no real significance to it, its just the way the directory detection logic works today. The double slashes on Linux get interpreted as a single one, and does not induce the behaviour in any way (same within Java's CLASSPATH parser, for which this entry is created)

Here's where the classpath entry is being built: /usr/lib/hadoop/libexec/hadoop-config.sh (you can vi this file on your installation)

Sourcing the variables that are being appended to CLASSPATH ('/*' is being appended to whose values) will reveal further that some of them come from the file /usr/lib/hadoop/libexec/hadoop-layout.sh (you can vi this as well) wherein they are set to './'.

As a result the concatenation operations on these strings produce the logic:

"/usr/lib/hadoop" (base) + "/" (separator) + "./" (specific sub-dir path) + "/*" (wildcard suffix)

The output of such a thing would thereby appear as "/usr/lib/hadoop/.//*".

Similar things apply to the non wildcard question, where we look to just add the directory itself onto the classpath.

Understanding-wise this is no different in evaluation than the reduced form of:

/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/*

Hope this helps in reducing the confusion.

View solution in original post

2 REPLIES 2

avatar
Mentor
There's no real significance to it, its just the way the directory detection logic works today. The double slashes on Linux get interpreted as a single one, and does not induce the behaviour in any way (same within Java's CLASSPATH parser, for which this entry is created)

Here's where the classpath entry is being built: /usr/lib/hadoop/libexec/hadoop-config.sh (you can vi this file on your installation)

Sourcing the variables that are being appended to CLASSPATH ('/*' is being appended to whose values) will reveal further that some of them come from the file /usr/lib/hadoop/libexec/hadoop-layout.sh (you can vi this as well) wherein they are set to './'.

As a result the concatenation operations on these strings produce the logic:

"/usr/lib/hadoop" (base) + "/" (separator) + "./" (specific sub-dir path) + "/*" (wildcard suffix)

The output of such a thing would thereby appear as "/usr/lib/hadoop/.//*".

Similar things apply to the non wildcard question, where we look to just add the directory itself onto the classpath.

Understanding-wise this is no different in evaluation than the reduced form of:

/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/*

Hope this helps in reducing the confusion.

avatar
New Contributor
Thanks Harsh! Nice explanation!