Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

TestDFSIO in latest HDP

avatar
Rising Star

Hi,

I am trying to run TestDFSIO performance tests on my cluster and trying to do as below.

Logged into the name node machine

Logged in as hdfs user

/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO -write -nrFiles 100 -fileSize 100 TestDFSIO Read Test hadoop jar

After running the above test, I am seeing a permission denied error. Not sure what could be the reason for it as I am running as hdfs user, but still it doesnt run.

bash: /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar: Permission denied

Any idea what could be the reason?

Thanks in advance.

1 ACCEPTED SOLUTION

avatar

The permission denied error is coming from bash, not Hadoop. Is that the full command that you were running? If so, then you ended up trying to execute the jar file directly. Since the jar file won't have the execute bit on in its permissions, this gets reported as permission denied. Instead, you would need to run the jar through the "hadoop jar" command.

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar

View solution in original post

4 REPLIES 4

avatar

The permission denied error is coming from bash, not Hadoop. Is that the full command that you were running? If so, then you ended up trying to execute the jar file directly. Since the jar file won't have the execute bit on in its permissions, this gets reported as permission denied. Instead, you would need to run the jar through the "hadoop jar" command.

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar

avatar
Rising Star

My bad. I forgot to start the command with "yarn jar". Apologies. Its working fine! Thanks

avatar
Master Mentor

@Greenhorn Techie

ls -l /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar

avatar
Expert Contributor

Run the jar with "hadoop jar" or "yarn jar"

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......... yarn jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......