Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

TestDFSIO in latest HDP

Solved Go to solution
Highlighted

TestDFSIO in latest HDP

Hi,

I am trying to run TestDFSIO performance tests on my cluster and trying to do as below.

Logged into the name node machine

Logged in as hdfs user

/usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO -write -nrFiles 100 -fileSize 100 TestDFSIO Read Test hadoop jar

After running the above test, I am seeing a permission denied error. Not sure what could be the reason for it as I am running as hdfs user, but still it doesnt run.

bash: /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar: Permission denied

Any idea what could be the reason?

Thanks in advance.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: TestDFSIO in latest HDP

The permission denied error is coming from bash, not Hadoop. Is that the full command that you were running? If so, then you ended up trying to execute the jar file directly. Since the jar file won't have the execute bit on in its permissions, this gets reported as permission denied. Instead, you would need to run the jar through the "hadoop jar" command.

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar

View solution in original post

4 REPLIES 4
Highlighted

Re: TestDFSIO in latest HDP

The permission denied error is coming from bash, not Hadoop. Is that the full command that you were running? If so, then you ended up trying to execute the jar file directly. Since the jar file won't have the execute bit on in its permissions, this gets reported as permission denied. Instead, you would need to run the jar through the "hadoop jar" command.

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar

View solution in original post

Highlighted

Re: TestDFSIO in latest HDP

My bad. I forgot to start the command with "yarn jar". Apologies. Its working fine! Thanks

Highlighted

Re: TestDFSIO in latest HDP

@Greenhorn Techie

ls -l /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar

Highlighted

Re: TestDFSIO in latest HDP

Expert Contributor

Run the jar with "hadoop jar" or "yarn jar"

hadoop jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......... yarn jar /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO .......

Don't have an account?
Coming from Hortonworks? Activate your account here