Support Questions
Find answers, ask questions, and share your expertise

mapreduce.cluster.local.dir in Mapreduce

mapreduce.cluster.local.dir in Mapreduce

Explorer

Hi, I have an SSD and I want to use them for temporary mapreduce files, I added a parameter in "mapreduce.cluster.local.dir" in "Custom mapred-site", but how do I know that this is the directory used in Mapreduce. I turned on debug Log, but there is also no information whether this directory is used.

env:

HDP-2.5.0.0

Ambari - 2.4.2.0

OS - RHEL 6.7

Sergey.

3 REPLIES 3
Highlighted

Re: mapreduce.cluster.local.dir in Mapreduce

Explorer

You have to ssh to your slave node(s) and check manually.

Highlighted

Re: mapreduce.cluster.local.dir in Mapreduce

Explorer

sar and iostat utility shows 0. How else can I check?

Highlighted

Re: mapreduce.cluster.local.dir in Mapreduce

Guru

@Sergey Paramoshkin, you can look for Mapreduce application's config detail to figure out whether the directory you added is being used or not.

You can use resource manager ui ( if application is running ) or job history server ui ( if application is finished) to check the value of mapreduce.cluster.local.dir property.

* Go to the node manager log for AM.

* On left hand side, you will find a table, under tool section, find a "configuration" link.

* click on this link, it will show all the configuration used by mapreduce application. There, look for mapreduce.cluster.local.dir to validate if it picked up correct configuration.