Support Questions
Find answers, ask questions, and share your expertise

What is functional difference between "mapreduce.cluster.temp.dir" and "mapreduce.cluster.local.dir"?

Cloudera Employee

From, https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-de... I understand that "mapreduce.cluster.local.dir" is defined to store intermediate data from mapreduce operation. But, the description of "mapreduce.cluster.temp.dir" was minimal to understand what kind of temporary data is written in there. Would it be cache and temp jars sort of data?

Thank you for your time.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: What is functional difference between "mapreduce.cluster.temp.dir" and "mapreduce.cluster.local.dir"?

Super Guru
@kkanchu

You are reading defaults for MRV1. With YARN/MRV2 mapreduce.cluster.local.dir has been replaced by yarn.nodemanager.local-dirs

This property uses your local disk for storing temporary files. I have not tried mapreduce.cluster.temp.dir but it seems to me the difference is that this is a location in your HDFS and not local file system. You can try running a small sample job and see the difference.

View solution in original post

2 REPLIES 2

Re: What is functional difference between "mapreduce.cluster.temp.dir" and "mapreduce.cluster.local.dir"?

Super Guru
@kkanchu

You are reading defaults for MRV1. With YARN/MRV2 mapreduce.cluster.local.dir has been replaced by yarn.nodemanager.local-dirs

This property uses your local disk for storing temporary files. I have not tried mapreduce.cluster.temp.dir but it seems to me the difference is that this is a location in your HDFS and not local file system. You can try running a small sample job and see the difference.

View solution in original post

Re: What is functional difference between "mapreduce.cluster.temp.dir" and "mapreduce.cluster.local.dir"?

Cloudera Employee

@mqureshi thank you for the reply. Yes, "mapreduce.cluster.temp.dir" seems to be legacy property which is no longer available in 2.x.