question MapReduce fails with benchmarking in Archives of Support Questions (Read Only)

MapReduce fails with benchmarking

Mark_Heydenrych — Mon, 25 Apr 2016 14:49:50 GMT

I'm currently running benchmarks on a newly installed HDP cluster, running HDP 2.4. I have run the following benchmarks so far:

TestDFSIO

Teragen/Terasort/Teravalidate

MRBench

These all give me essentially the same error:

Diagnostics: ExitCodeException exitCode=2:

gzip: /foo/bar/yarn/local/filecache/14_tmp/tmp_mapreduce.tar.gz: unexpected end of file

tar: This does not look like a tar archive

tar: Exiting with failure status due to previous errors

I assume that the mapreduce jar is copied into a temp folder for the job. The original mapreduce jar is a valid tar, and I have extracted it with 7zip showing it has all the expected contents.

Has anybody seen this error before? Any guidance on how I might go about solving it?

Thanks in advance.

Re: MapReduce fails with benchmarking

aervits — Mon, 25 Apr 2016 15:35:49 GMT

I would review the mapreduce section in the manual installation guide, i always refer to it even if i use automated install http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/configure_yarn_and_mapreduce.html

Re: MapReduce fails with benchmarking

Mark_Heydenrych — Mon, 25 Apr 2016 16:05:58 GMT

That was the issue, thank you. The local version of the tarball was fine, but the one in DFS had zero size. Uploading the local version gets my MapReduce working as expected.