TestDFSIO in latest HDP

[#1]

Hi,

I am trying to run TestDFSIO performance tests on my cluster and trying to do as below.

Logged into the name node machine

Logged in as hdfs user

/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO -write -nrFiles 100 -fileSize 100 TestDFSIO Read Test hadoop jar

After running the above test, I am seeing a permission denied error. Not sure what could be the reason for it as I am running as hdfs user, but still it doesnt run.

bash: /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar: Permission denied

Any idea what could be the reason?

Thanks in advance.

[#2 · Accepted solution]

The permission denied error is coming from bash, not Hadoop. Is that the full command that you were running? If so, then you ended up trying to execute the jar file directly. Since the jar file won't have the execute bit on in its permissions, this gets reported as permission denied. Instead, you would need to run the jar through the "hadoop jar" command.

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar

[#3]

My bad. I forgot to start the command with "yarn jar". Apologies. Its working fine! Thanks

[#4]

@Greenhorn Techie

ls -l /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar

[#5]

Run the jar with "hadoop jar" or "yarn jar"

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......... yarn jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......