Member since
11-02-2022
2
Posts
0
Kudos Received
0
Solutions
11-02-2022
09:32 PM
hi Asim, Thanks for the reply. I can see the log using the command while the hadoop cluster is running. After I reboot the cluster, the history logs disappead on the page. The config is as below: <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <!-- 历史服务器端地址 --> <property> <name>mapreduce.jobhistory.address</name> <value>hadoop201:10020</value> </property> <!-- 历史服务器web端地址 --> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>hadoop201:19888</value> </property> <property> <name>mapreduce.jobhistory.done-dir</name> <value>/opt/module/hadoop-3.1.3/logs/his_log/done</value> <description>MR JobHistory Server管理的日志的存放位置,默认:/mr-history/done</description> </property> <property> <name>mapreduce.jobhistory.intermediate-done-dir</name> <value>/opt/module/hadoop-3.1.3/logs/his_log</value> <description>MapReduce作业产生的日志存放位置,默认值:/mr-history/tmp</description> </property> <property> <name>yarn.app.mapreduce.am.staging-dir</name> <value>/opt/module/hadoop-3.1.3/logs/mr-stage-his</value> <description></description> </property> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds</name> <value>3600</value> </property> <property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/opt/module/hadoop-3.1.3/logs/resource_manager_logs</value> </property> <!-- 设置日志聚集服务器地址 --> <property> <name>yarn.log.server.url</name> <value>http://hadoop201:19888/jobhistory/logs</value> </property> <!-- 设置日志保留时间为7天 --> <property> <name>yarn.log-aggregation.retain-seconds</name> <value>5184000</value> </property> much appreciated for the help.
... View more
11-02-2022
01:29 AM
I run some mr tasks and can see the task status on yarn web. But when I reboot the hadoop cluster, the task records disappear. all the necessary config have been set and the jobhistory app was running too. Any suggestion for debug?
... View more
Labels:
- Labels:
-
Apache Hadoop