Question about map output

Kevin_DElia — Fri, 16 Sep 2022 09:07:03 GMT

I'm looking for clarification on something I've read in Hadoop: The Definitive Guide. It states that map output is local and not in HDFS. But the map task runs on the node where the data resides (usually) and that is HDFS, correct? Or is it the case that the I/O done by the map is standard Java I/O and not something like hadoop fs -put?

Thanks in advance to all who answer.

Re: Question about map output

GautamG — Mon, 08 Sep 2014 07:14:17 GMT

The map task's local output is not stored within HDFS, rather in temporary
directories on that specific node (see property mapreduce.cluster.local.dir)
written using standard file I/O

https://hadoop.apache.org/docs/r2.2.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

question Question about map output in Archives of Support Questions (Read Only)

Question about map output

Re: Question about map output