Unable to start mapreduce2's History Server

[#1]

when I debug yarn for a good performance on my cluster. The history server cannot be started ,and I cannot find logs in /var/log/hadoop or /var/log/hadoop-mapreduce.now error starting the history server .Only ambari-web shows some problems ,like this:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 182, in <module>
    HistoryServer().execute()
  File "/usr/lib/python2.7/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 92, in start
    self.configure(env) # FOR SECURITY
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 55, in configure
    yarn(name="historyserver")
  File "/usr/lib/python2.7/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
    return fn(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py", line 98, in yarn
    mode=0777
  File "/usr/lib/python2.7/site-packages/resource_management/core/base.py", line 125, in __new__
    env.resources[r_type][name] = obj
  File "/usr/lib/python2.7/site-packages/resource_management/libraries/script/config_dictionary.py", line 81, in __getattr__
    raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'mapreduce.jobhistory.done-dir' was not found in configurations dictionary!

[#2 · Accepted solution]

@Mon key

Hi, Can you cross check if you have below property in Mapreduce2 -> config section on Ambari UI?

mapreduce.jobhistory.done-dir = /mr-history/done

If not and try adding this property under "Advanced mapred-site" section, the default value of this parameter is "/mr-history/done"

[#3]

I love you so much.. but I want to know how it happens . I don't change other arguments except yarn's .

[#4]

@Mon key

Frankly i don't know how it got removed from your cluster 🙂 but this property comes default when you install History server. Also please accept the answer if that helped you to resolve this issue.

[#5]

hi,

In my case, the permission on the following folder caused the issue.

/hadoop/mapreduce/jhs/mr-jhs-state

The following log message at /var/log/hadoop-mapreduce/mapred/mapred-mapred-historyserver-iml6.log helped me solve it. For some reason the files were owned by Yarn, when I given permission to the group I could see the files were rewritten by mapred.

IO error: /hadoop/mapreduce/jhs/mr-jhs-state/LOCK: Permission denied

~BASU