question Re: TestDFSIO in latest HDP in Archives of Support Questions (Read Only)

TestDFSIO in latest HDP

techiegreenhorn — Fri, 29 Jan 2016 03:36:42 GMT

Hi,

I am trying to run TestDFSIO performance tests on my cluster and trying to do as below.

Logged into the name node machine

Logged in as hdfs user

/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO -write -nrFiles 100 -fileSize 100 TestDFSIO Read Test hadoop jar

After running the above test, I am seeing a permission denied error. Not sure what could be the reason for it as I am running as hdfs user, but still it doesnt run.

bash: /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar: Permission denied

Any idea what could be the reason?

Thanks in advance.

Re: TestDFSIO in latest HDP

cnauroth — Fri, 29 Jan 2016 04:02:52 GMT

The permission denied error is coming from bash, not Hadoop. Is that the full command that you were running? If so, then you ended up trying to execute the jar file directly. Since the jar file won't have the execute bit on in its permissions, this gets reported as permission denied. Instead, you would need to run the jar through the "hadoop jar" command.

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar

Re: TestDFSIO in latest HDP

nsabharwal — Fri, 29 Jan 2016 04:07:47 GMT

@Greenhorn Techie

ls -l /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar

Re: TestDFSIO in latest HDP

techiegreenhorn — Fri, 29 Jan 2016 04:13:41 GMT

My bad. I forgot to start the command with "yarn jar". Apologies. Its working fine! Thanks

Re: TestDFSIO in latest HDP

rajsyrus — Tue, 03 May 2016 13:17:52 GMT

Run the jar with "hadoop jar" or "yarn jar"

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......... yarn jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......