question Re: Benchmark test ouput file permission denied (TestDFSIO_results.log ) in Archives of Support Questions (Read Only)

Benchmark test ouput file permission denied (TestDFSIO_results.log )

JB0000000000001 — Mon, 22 Jan 2018 16:21:35 GMT

Dear Community,
I am running a well-known benchmark test to measure the I/O of my cluster using the hadoop-test-2.6.0-mr1-cdh5.13.0.jar using the TestDFSIO test. Normally the ouput should be written to TestDFSIO_results.log in the current folder of the bash (not hdfs) where I run the command. THis is in my case just my home folder.
I am running the command as follows:
sudo -u hdfs hadoop jar /opt/cloudera/parcels/CDH/jars/hadoop-test-2.6.0-mr1-cdh5.13.0.jar TestDFSIO -zrite -nrFiles 2 -size 100MB.
As it seems I need to run the job with the hdfs user, I have changed the permission of the default output file to 777 and set the owner and group to hdfs, but also this does not help (also previously when my own user was the owner it did not work).

Thanks for any help!!

Re: Benchmark test ouput file permission denied (TestDFSIO_results.log )

JB0000000000001 — Tue, 06 Feb 2018 10:32:16 GMT

I just solved this issue by starting the command while I was in /tmp. The problem was the user of the hadoop jar job (hdfs) did not have sufficient rights to write to the local system. When executing from /tmp/ the output file with the results was correctly produced.

Hope this can help others in the future!

Re: Benchmark test ouput file permission denied (TestDFSIO_results.log )

doonaks — Thu, 22 Aug 2019 16:03:24 GMT

The directory /benchmarks is owned as "hdfs:superuser"

If you run the benchmark job with user hdfs, it wont run as hdfs is a banned user in Yarn configuration for container executor snippet.

what you need to do is to change the permission of "/benchmarks" as your custom user using hdfs token. Then you would be able to run the benchmarks job without any issue.

------

hadoop fs -chown -R user02:user02 /benchmarks

hadoop fs -ls /benchmarks
Found 1 items
drwxr-xr-x - user02 user02 0 2019-08-22 15:55 /benchmarks/TestDFSIO

------