Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

What is the significance of /.//* in hadoop classpath?

avatar
Visitor

[cloudera@quickstart ~]$ hadoop classpath
/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*

 

 

The above is what I get when I use hadoop classpath. I am wondering what is the meaning of

/usr/lib/hadoop/.//*

and also

/usr/lib/hadoop-hdfs/./

 

I know that * matches anything and . stands for the current directory. But i can't make any sense of them being used like that together with double slashes (//).

1 ACCEPTED SOLUTION

avatar
Mentor
There's no real significance to it, its just the way the directory detection logic works today. The double slashes on Linux get interpreted as a single one, and does not induce the behaviour in any way (same within Java's CLASSPATH parser, for which this entry is created)

Here's where the classpath entry is being built: /usr/lib/hadoop/libexec/hadoop-config.sh (you can vi this file on your installation)

Sourcing the variables that are being appended to CLASSPATH ('/*' is being appended to whose values) will reveal further that some of them come from the file /usr/lib/hadoop/libexec/hadoop-layout.sh (you can vi this as well) wherein they are set to './'.

As a result the concatenation operations on these strings produce the logic:

"/usr/lib/hadoop" (base) + "/" (separator) + "./" (specific sub-dir path) + "/*" (wildcard suffix)

The output of such a thing would thereby appear as "/usr/lib/hadoop/.//*".

Similar things apply to the non wildcard question, where we look to just add the directory itself onto the classpath.

Understanding-wise this is no different in evaluation than the reduced form of:

/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/*

Hope this helps in reducing the confusion.

View solution in original post

2 REPLIES 2

avatar
Mentor
There's no real significance to it, its just the way the directory detection logic works today. The double slashes on Linux get interpreted as a single one, and does not induce the behaviour in any way (same within Java's CLASSPATH parser, for which this entry is created)

Here's where the classpath entry is being built: /usr/lib/hadoop/libexec/hadoop-config.sh (you can vi this file on your installation)

Sourcing the variables that are being appended to CLASSPATH ('/*' is being appended to whose values) will reveal further that some of them come from the file /usr/lib/hadoop/libexec/hadoop-layout.sh (you can vi this as well) wherein they are set to './'.

As a result the concatenation operations on these strings produce the logic:

"/usr/lib/hadoop" (base) + "/" (separator) + "./" (specific sub-dir path) + "/*" (wildcard suffix)

The output of such a thing would thereby appear as "/usr/lib/hadoop/.//*".

Similar things apply to the non wildcard question, where we look to just add the directory itself onto the classpath.

Understanding-wise this is no different in evaluation than the reduced form of:

/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/*

Hope this helps in reducing the confusion.

avatar
Visitor
Thanks Harsh! Nice explanation!