question Re: ClassNotFoundException: TestDFSIO in Archives of Support Questions (Read Only)

ClassNotFoundException: TestDFSIO

yongie — Fri, 16 Sep 2022 13:49:35 GMT

Hi, I have been trying to run TestDFSIO, but it always give me ClassNotFoundException: TestDFSIO, I tried to run using /usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-client-test.jar, /usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-native-client-test.jar, /usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar, /usr/hdp/current/hadoop-client/hadoop-common-tests.jar, /usr/hdp/3.0.0.0-1634/hadoop/hadoop-common-tests.jar, /usr/hdp/3.0.0.0-1634/hadoop-hdfs/hadoop-hdfs-tests.jar, all of them gave me same output, anything I did wrong?

Re: ClassNotFoundException: TestDFSIO

jsensharma — Tue, 23 Oct 2018 10:24:37 GMT

@Ronnie 10

How are you running it ?

Can you share the exact command?

Example:

# su - hdfs
# hadoop jar /usr/hdp/3.0.0.0-1634/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100000

Re: ClassNotFoundException: TestDFSIO

yongie — Tue, 23 Oct 2018 10:58:56 GMT

I try run again with

sudo -u hdfs hadoop jar hadoop-hdfs-client-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100000

it still showing

Exception in thread "main" java.lang.ClassNotFoundException: TestDFSIO

Re: ClassNotFoundException: TestDFSIO

jsensharma — Tue, 23 Oct 2018 11:23:43 GMT

@Ronnie 10

Your JAR name is different. "hadoop-hdfs-client-tests.jar" (wrong jar) and I am using "hadoop-mapreduce-client-jobclient-tests.jar" (correct JAR)

Your command:

# sudo -u hdfs hadoop jar hadoop-hdfs-client-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100000

My Command:

# cd /usr/hdp/3.0.0.0-1634/hadoop-mapreduce
# sudo -u hdfs hadoop jar hadoop-mapreduce-client-jobclient-tests.jar  TestDFSIO -write -nrFiles 10 -fileSize 100000

You can chekc if your JAR contains that class or not ass following:

# /usr/jdk64/jdk1.8.0_112/bin/jar -tvf /usr/hdp/3.0.0.0-1634/hadoop-hdfs/hadoop-hdfs-client-tests.jar | grep TestDFSIO

Not Found

But if you use the correct JAR then you can see it.

# /usr/jdk64/jdk1.8.0_112/bin/jar -tvf /usr/hdp/3.0.0.0-1634/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-tests.jar | grep TestDFSIO
  2323 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$ByteMultiple.class
  2974 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$ReadMapper.class
  4215 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$RandomReadMapper.class
  3148 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$TruncateMapper.class
  3233 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$WriteMapper.class
  4014 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$IOStatMapper.class
  1086 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$1.class
 24732 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO.class
  3078 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$AppendMapper.class
  1895 Thu Jul 12 20:36:52 UTC 2018 org/apache/hadoop/fs/TestDFSIO$TestType.class

Re: ClassNotFoundException: TestDFSIO

yongie — Wed, 24 Oct 2018 09:43:16 GMT

@Jay Kumar SenSharma

Thanks for the reply, I have found the jar file to run the TestDFSio!

It is inside

/usr/hdp/current/hadoop-mapreduce-client