ClassNotFoundException: TestDFSIO

[#1]

Hi, I have been trying to run TestDFSIO, but it always give me ClassNotFoundException: TestDFSIO, I tried to run using /usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-client-test.jar, /usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-native-client-test.jar, /usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar, /usr/hdp/current/hadoop-client/hadoop-common-tests.jar, /usr/hdp/3.0.0.0-1634/hadoop/hadoop-common-tests.jar, /usr/hdp/3.0.0.0-1634/hadoop-hdfs/hadoop-hdfs-tests.jar, all of them gave me same output, anything I did wrong?

[#2]

@Ronnie 10

How are you running it ?

Can you share the exact command?

Example:

# su - hdfs
# hadoop jar /usr/hdp/3.0.0.0-1634/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100000

.

[#3]

I try run again with

sudo -u hdfs hadoop jar hadoop-hdfs-client-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100000

it still showing

Exception in thread "main" java.lang.ClassNotFoundException: TestDFSIO

[#4 · Accepted solution]

@Ronnie 10

Your JAR name is different. "hadoop-hdfs-client-tests.jar" (wrong jar) and I am using "hadoop-mapreduce-client-jobclient-tests.jar" (correct JAR)

Your command:

# sudo -u hdfs hadoop jar hadoop-hdfs-client-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100000

My Command:

# cd /usr/hdp/3.0.0.0-1634/hadoop-mapreduce
# sudo -u hdfs hadoop jar hadoop-mapreduce-client-jobclient-tests.jar  TestDFSIO -write -nrFiles 10 -fileSize 100000

.

You can chekc if your JAR contains that class or not ass following:

# /usr/jdk64/jdk1.8.0_112/bin/jar -tvf /usr/hdp/3.0.0.0-1634/hadoop-hdfs/hadoop-hdfs-client-tests.jar | grep TestDFSIO

Not Found

.

But if you use the correct JAR then you can see it.

# /usr/jdk64/jdk1.8.0_112/bin/jar -tvf /usr/hdp/3.0.0.0-1634/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-tests.jar | grep TestDFSIO
  2323 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$ByteMultiple.class
  2974 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$ReadMapper.class
  4215 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$RandomReadMapper.class
  3148 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$TruncateMapper.class
  3233 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$WriteMapper.class
  4014 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$IOStatMapper.class
  1086 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$1.class
 24732 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO.class
  3078 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$AppendMapper.class
  1895 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$TestType.class

.

[#5]

@Jay Kumar SenSharma

Thanks for the reply, I have found the jar file to run the TestDFSio!

It is inside

/usr/hdp/current/hadoop-mapreduce-client