Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Distributed I/O Benchmark of HDFS

Solved Go to solution
Highlighted

Distributed I/O Benchmark of HDFS


hi all


DFSIO is a built-in benchmark tool for HDFS I/O test.

we created the Base Directory to Store Test Results

/benchmarks/TestDFSIO


and when we run the test we get:


 hadoop jar /usr/hdp/2.6.4.0-91/spark2/jars/hadoop-mapreduce-client-jobclient-2.7.3.2.6.4.0-91.jar TestDFSIO -write -nrFiles 16 -fileSize 1GB -resFile /tmp/USER-dfsio-write.txt
19/03/31 05:39:57 INFO fs.TestDFSIO: TestDFSIO.1.8
19/03/31 05:39:57 INFO fs.TestDFSIO: nrFiles = 16
19/03/31 05:39:57 INFO fs.TestDFSIO: nrBytes (MB) = 1024.0
19/03/31 05:39:57 INFO fs.TestDFSIO: bufferSize = 1000000
19/03/31 05:39:57 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO
19/03/31 05:39:59 INFO fs.TestDFSIO: creating control file: 1073741824 bytes, 16 files
java.io.IOException: Permission denied: user=root, access=WRITE, inode="/benchmarks/TestDFSIO/io_control/in_file_test_io_0":hdfs:hdfs:drwxr-xr-x


where we are wrong?

Michael-Bronson
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Distributed I/O Benchmark of HDFS

@Michael Bronson, Permission issue :) Either run this command with hdfs user or change the ownership of /benchmarks/TestDFSIO to root.

java.io.IOException: Permission denied: user=root, access=WRITE, inode="/benchmarks/TestDFSIO/io_control/in_file_test_io_0":hdfs:hdfs:drwxr-xr-x 

1 REPLY 1

Re: Distributed I/O Benchmark of HDFS

@Michael Bronson, Permission issue :) Either run this command with hdfs user or change the ownership of /benchmarks/TestDFSIO to root.

java.io.IOException: Permission denied: user=root, access=WRITE, inode="/benchmarks/TestDFSIO/io_control/in_file_test_io_0":hdfs:hdfs:drwxr-xr-x