Not Valid Jar Error

[#1]

Hi .

While running wordcount jar I am getting error"Not a valid jar"

[hdfs@sandbox /]$ hadoop fs -ls

Found 8 items

drwx------ - root hdfs 0 2016-08-12 00:21 .Trash

drwxr-xr-x - root hdfs 0 2016-08-01 21:31 .hiveJars

drwx------ - hdfs hdfs 0 2016-08-12 19:42 .staging

-rw-r--r-- 3 admin hdfs 1024 2016-08-14 19:24 count.txt

drwxr-xr-x - hdfs hdfs 0 2016-08-12 18:38 emp

-rw-r--r-- 3 admin hdfs 4096 2016-08-14 19:24 wordcount.jar

[hdfs@sandbox /]$ hadoop jar wordcount.jar WordCount count.txt wordcountoutput

WARNING: Use "yarn jar" to launch YARN applications. Not a valid JAR: /wordcount.jar

I compiled my program with hadoop-common-2.3.0.jar

Hadoop and Java version as below

Hadoop 2.7.1.2.3.2.0-2950

java version "1.7.0_91"

[#2]

@Tech Guy

Your jar appears to be in hdfs. You cannot have jar inside hdfs. It needs to be in your local file system. Put it in local file system and run again. It should work.

[#3]

It can be in hdfs or local fs. You just need to have them referenced properly and have the proper privileges.

[#4]

Hi @Constantin Stanca, @Tech Guy

I am pretty sure, the jar file is supposed to be in local file system. Unless, you are aware of things getting changed in new versions. Please see the following link.

http://stackoverflow.com/questions/20333135/how-to-execute-hadoop-jar-from-hdfs-filesystem

[#5]

@mqureshi

Technically, you are correct. Since I did not know the reason for storing the jar file in hdfs, I wanted to provide a "either way" solution. For example, in large development teams, I used a trick for cases where needed to store the jar files in HDFS as such they can be easily accessed between multiple clients including Hive as UDFs. It was something like in the reference you provided, but you are correct, it will still pick it from the local file system, however, it will provide a centralized location which can be used as a target for build artifacts and shared across multiple developers in a team.

hadoop fs -copyToLocal hdfs:///home/usr/jar/myjar.jar /tmp/myjar.jar && hadoop jar /tmp/myjar.jar com.test.TestMain 

[#6]

@Constantin Stanca, @mqureshi,

I tried to run in local file system as below

[hdfs@sandbox LogFiles]$ hadoop jar wordcount.jar WordCount count.txt /Out

Not valid error gone but got "Unsupported major.minor version 52.0" Error.

I compiled my program with hadoop-common-2.7.1.jar

Hadoop and Java version as below

Hadoop 2.7.1.2.3.2.0-2950

java version "1.7.0_91"

Please suggest.

[#7]

@mqureshi

I tried to run in local file system as below

[hdfs@sandbox LogFiles]$ hadoop jar wordcount.jar WordCount count.txt /Out

Not valid error gone but got "Unsupported major.minor version 52.0" Error.

I compiled my program with hadoop-common-2.7.1.jar

Hadoop and Java version as below

Hadoop 2.7.1.2.3.2.0-2950

java version "1.7.0_91"

Please suggest.

[#8]

@Tech Guy compile your jar with Java 1.7 and run the job.

[#9]

Thanks. It worked

[#10]

@Tech Guy

The message is misleading. You need to provide a path for your input and output files and make sure that your user has privileges on those folders. You should also try to place the jar locally as @mqureshi suggested, which is probably the easiest, otherwise you would need to specify a full hdfs path to the jar file.

For example, you could run the command as it follows:

hadoop jar /home/joe/wordcount.jar WordCount /user/joe/wordcount/input /user/joe/wordcount/output

If the response is helpful, vote/accept answer.