Support Questions

Find answers, ask questions, and share your expertise

Unable to start mapreduce2's History Server

avatar
Explorer

when I debug yarn for a good performance on my cluster. The history server cannot be started ,and I cannot find logs in /var/log/hadoop or /var/log/hadoop-mapreduce.now error starting the history server .Only ambari-web shows some problems ,like this:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 182, in <module>
    HistoryServer().execute()
  File "/usr/lib/python2.7/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 92, in start
    self.configure(env) # FOR SECURITY
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 55, in configure
    yarn(name="historyserver")
  File "/usr/lib/python2.7/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
    return fn(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py", line 98, in yarn
    mode=0777
  File "/usr/lib/python2.7/site-packages/resource_management/core/base.py", line 125, in __new__
    env.resources[r_type][name] = obj
  File "/usr/lib/python2.7/site-packages/resource_management/libraries/script/config_dictionary.py", line 81, in __getattr__
    raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'mapreduce.jobhistory.done-dir' was not found in configurations dictionary!
1 ACCEPTED SOLUTION

avatar
Super Guru

@Mon key

Hi, Can you cross check if you have below property in Mapreduce2 -> config section on Ambari UI?

mapreduce.jobhistory.done-dir = /mr-history/done

If not and try adding this property under "Advanced mapred-site" section, the default value of this parameter is "/mr-history/done"

View solution in original post

4 REPLIES 4

avatar
Super Guru

@Mon key

Hi, Can you cross check if you have below property in Mapreduce2 -> config section on Ambari UI?

mapreduce.jobhistory.done-dir = /mr-history/done

If not and try adding this property under "Advanced mapred-site" section, the default value of this parameter is "/mr-history/done"

avatar
Explorer

I love you so much.. but I want to know how it happens . I don't change other arguments except yarn's .

avatar
Super Guru

@Mon key

Frankly i don't know how it got removed from your cluster 🙂 but this property comes default when you install History server. Also please accept the answer if that helped you to resolve this issue.

avatar

hi,

In my case, the permission on the following folder caused the issue.

/hadoop/mapreduce/jhs/mr-jhs-state

The following log message at /var/log/hadoop-mapreduce/mapred/mapred-mapred-historyserver-iml6.log helped me solve it. For some reason the files were owned by Yarn, when I given permission to the group I could see the files were rewritten by mapred.

IO error: /hadoop/mapreduce/jhs/mr-jhs-state/LOCK: Permission denied

~BASU