Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

History Server intermitent shutdown.

Solved Go to solution
Highlighted

History Server intermitent shutdown.

New Contributor

We are facing Mapreduce2 > History Server intermitent shutdown.
Once we start the process it works for some time and then if throws the below log:

I saw in ambari History Server heap size = 900Mb, increase it to 1000Mb but still the same.

Please let me know any comment or if any detail is required to troubleshoot this.

2017-10-05 18:57:01,506 ERROR mortbay.log (Slf4jLog.java:warn(87)) - handle failed
2017-10-05 19:06:26,909 ERROR mortbay.log (Slf4jLog.java:warn(87)) - Error for /jmx
2017-10-05 19:00:19,549 INFO  delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:createPassword(385)) - Creating password for identifier: owner=srvc_egi_hdev, renewer=yarn, realUser=oozie/hc4t02046.itcs.hpecorp.net@EAHPEDEV.ITCS.HPECORP.NET, issueDate=1507230017382, maxDate=1507834817382, sequenceNumber=1183674, masterKeyId=322, currentKey: 322
2017-10-05 18:57:26,058 WARN  server.AuthenticationFilter (AuthenticationFilter.java:doFilter(530)) - AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature
2017-10-05 19:16:11,695 INFO  delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:createPassword(385)) - Creating password for identifier: owner=srvc_egi_hdev, renewer=yarn, realUser=oozie/hc4t02046.itcs.hpecorp.net@EAHPEDEV.ITCS.HPECORP.NET, issueDate=1507230971695, maxDate=1507835771695, sequenceNumber=1183675, masterKeyId=322, currentKey: 322
2017-10-05 19:16:11,694 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51)) - Thread Thread[Timer-1,5,main] threw an Error.  Shutting down now...
java.lang.OutOfMemoryError: Java heap space
2017-10-05 19:15:00,875 FATAL ipc.Server (Server.java:doRunLoop(870)) - Bug in read selector!
java.lang.OutOfMemoryError: Java heap space
2017-10-05 19:14:47,223 ERROR mortbay.log (Slf4jLog.java:warn(87)) - /jmx?qry=java.lang:type=OperatingSystem
java.lang.OutOfMemoryError: Java heap space
2017-10-05 19:16:11,698 WARN  server.AuthenticationFilter (AuthenticationFilter.java:doFilter(530)) - AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature
2017-10-05 19:16:11,698 WARN  server.AuthenticationFilter (AuthenticationFilter.java:doFilter(530)) - AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature
2017-10-05 19:14:42,796 ERROR mortbay.log (Slf4jLog.java:warn(87)) - Error for /jobhistory
java.lang.OutOfMemoryError: Java heap space
2017-10-05 19:10:41,467 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51)) - Thread Thread[IPC Client (137225802) connection to hc4t02044.itcs.hpecorp.net/16.202.10.51:8020 from jhs/hc4t02252.itcs.hpecorp.net@EAHPEDEV.ITCS.HPECORP.NET,5,main] threw an Error.  Shutting down now...
java.lang.OutOfMemoryError: Java heap space
2017-10-05 19:16:11,703 INFO  util.ExitUtil (ExitUtil.java:halt(147)) - Halt with status -1 Message: HaltException
2017-10-05 19:09:22,111 ERROR mortbay.log (Slf4jLog.java:warn(87)) - handle failed
java.lang.OutOfMemoryError: Java heap space
2017-10-05 19:08:14,649 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51)) - Thread Thread[666280300@qtp-444127949-5,5,main] threw an Error.  Shutting down now...
java.lang.OutOfMemoryError: Java heap space
<br>
1 ACCEPTED SOLUTION

Accepted Solutions

Re: History Server intermitent shutdown.

New Contributor

Increased the History Server heap size to 2046MB fixed the issue.

Regards.

1 REPLY 1

Re: History Server intermitent shutdown.

New Contributor

Increased the History Server heap size to 2046MB fixed the issue.

Regards.

Don't have an account?
Coming from Hortonworks? Activate your account here