Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!
jesus-angel_mon
user rank
Explorer
My Accepted Solutions
Show All

Re: History Server intermitent shutdown.

by Explorer in Support Questions
‎10-10-2017 05:00 PM
‎10-10-2017 05:00 PM
Increased the History Server heap size to 2046MB fixed the issue. Regards. ... View more

History Server intermitent shutdown.

by Explorer in Support Questions
‎10-05-2017 07:27 PM
‎10-05-2017 07:27 PM
We are facing Mapreduce2 > History Server intermitent shutdown. Once we start the process it works for some time and then if throws the below log: I saw in ambari History Server heap size = 900Mb, increase it to 1000Mb but still the same. Please let me know any comment or if any detail is required to troubleshoot this. 2017-10-05 18:57:01,506 ERROR mortbay.log (Slf4jLog.java:warn(87)) - handle failed 2017-10-05 19:06:26,909 ERROR mortbay.log (Slf4jLog.java:warn(87)) - Error for /jmx 2017-10-05 19:00:19,549 INFO delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:createPassword(385)) - Creating password for identifier: owner=srvc_egi_hdev, renewer=yarn, realUser=oozie/hc4t02046.itcs.hpecorp.net@EAHPEDEV.ITCS.HPECORP.NET, issueDate=1507230017382, maxDate=1507834817382, sequenceNumber=1183674, masterKeyId=322, currentKey: 322 2017-10-05 18:57:26,058 WARN server.AuthenticationFilter (AuthenticationFilter.java:doFilter(530)) - AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature 2017-10-05 19:16:11,695 INFO delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:createPassword(385)) - Creating password for identifier: owner=srvc_egi_hdev, renewer=yarn, realUser=oozie/hc4t02046.itcs.hpecorp.net@EAHPEDEV.ITCS.HPECORP.NET, issueDate=1507230971695, maxDate=1507835771695, sequenceNumber=1183675, masterKeyId=322, currentKey: 322 2017-10-05 19:16:11,694 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51)) - Thread Thread[Timer-1,5,main] threw an Error. Shutting down now... java.lang.OutOfMemoryError: Java heap space 2017-10-05 19:15:00,875 FATAL ipc.Server (Server.java:doRunLoop(870)) - Bug in read selector! java.lang.OutOfMemoryError: Java heap space 2017-10-05 19:14:47,223 ERROR mortbay.log (Slf4jLog.java:warn(87)) - /jmx?qry=java.lang:type=OperatingSystem java.lang.OutOfMemoryError: Java heap space 2017-10-05 19:16:11,698 WARN server.AuthenticationFilter (AuthenticationFilter.java:doFilter(530)) - AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature 2017-10-05 19:16:11,698 WARN server.AuthenticationFilter (AuthenticationFilter.java:doFilter(530)) - AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature 2017-10-05 19:14:42,796 ERROR mortbay.log (Slf4jLog.java:warn(87)) - Error for /jobhistory java.lang.OutOfMemoryError: Java heap space 2017-10-05 19:10:41,467 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51)) - Thread Thread[IPC Client (137225802) connection to hc4t02044.itcs.hpecorp.net/16.202.10.51:8020 from jhs/hc4t02252.itcs.hpecorp.net@EAHPEDEV.ITCS.HPECORP.NET,5,main] threw an Error. Shutting down now... java.lang.OutOfMemoryError: Java heap space 2017-10-05 19:16:11,703 INFO util.ExitUtil (ExitUtil.java:halt(147)) - Halt with status -1 Message: HaltException 2017-10-05 19:09:22,111 ERROR mortbay.log (Slf4jLog.java:warn(87)) - handle failed java.lang.OutOfMemoryError: Java heap space 2017-10-05 19:08:14,649 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51)) - Thread Thread[666280300@qtp-444127949-5,5,main] threw an Error. Shutting down now... java.lang.OutOfMemoryError: Java heap space <br> ... View more

Re: Mysql server do not start : ambari-server star...

by Explorer in Support Questions
‎10-05-2017 06:56 PM
‎10-05-2017 06:56 PM
Everything indicates that many errors occurred during the preparation of the cluster and in the same way during the installation of HDP. We performed a re-image of everything and re-installation. Sandbox is up now. Thanks. ... View more

Re: Mysql server do not start : ambari-server star...

by Explorer in Support Questions
‎09-14-2017 08:39 PM
‎09-14-2017 08:39 PM
g4t7474_[root@g4t7474 ~]# chown mysql /var/run/mysqld/mysql.sock chown: invalid user: `mysql' g4t7474_[root@g4t7474 ~]# /usr/local/mysql/bin/safe_mysqld --user=mysql --skip-grant-tables bash: /usr/local/mysql/bin/safe_mysqld: No such file or directory g4t7474_[root@g4t7474 ~]# ls -ld /usr/local/mysql/bin/safe_mysqld ls: cannot access /usr/local/mysql/bin/safe_mysqld: No such file or directory g4t7474_[root@g4t7474 ~]# locate safe_mysqld g4t7474_[root@g4t7474 ~]# But the i was working with root: from: /etc/my.cnf # Disabling symbolic-links is recommended to prevent assorted security risks symbolic-links=0 #log-error=/var/log/mysqld.log #pid-file=/var/run/mysqld/mysqld.pid [mysqld] #datadir=/var/lib/mysql datadir=/opt/tools/mysql socket=/var/lib/mysql/mysql.sock #validate-password= off innodb_data_file_path = ibdata1:10M:autoextend [mysqladmin] user=root [client] user=mysql # Recommended in standard MySQL setup #sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pi ... View more

Re: Mysql server do not start : ambari-server star...

by Explorer in Support Questions
‎09-14-2017 06:06 PM
‎09-14-2017 06:06 PM
Same: g4t7474_[root@g4t7474 ~]# cd /tmp g4t7474_[root@g4t7474 tmp]# ln -s /var/mysql/mysql.sock mysql.sock g4t7474_[root@g4t7474 tmp]# service mysqld start MySQL Daemon failed to start. Starting mysqld: [FAILED] ... View more

Re: Mysql server do not start : ambari-server star...

by Explorer in Support Questions
‎09-14-2017 06:04 PM
‎09-14-2017 06:04 PM
g4t7474_[root@g4t7474 etc]# ls -ld /var/run/mysqld/mysqld.pid ls: cannot access /var/run/mysqld/mysqld.pid: No such file or directory g4t7474_[root@g4t7474 etc]# ls -ld /var/lib/mysqld/mysqld.pid ls: cannot access /var/lib/mysqld/mysqld.pid: No such file or directory g4t7474_[root@g4t7474 etc]# locate mysqld.pid g4t7474_[root@g4t7474 etc]# Any process hanging. ... View more

Re: Mysql server do not start : ambari-server star...

by Explorer in Support Questions
‎09-13-2017 01:49 PM
‎09-13-2017 01:49 PM
Hi Geoffrey, Thanks for your response, I tried but it didn't work. Please find the below details. mysql.sock is located on /var/lib/mysql/mysql.sock +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [root@g4t7474 etc]# locate mysql.sock /var/lib/mysql/mysql.sock +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ I tried +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [root@g4t7474 etc]# cd /tmp/ [root@g4t7474 tmp]# ls -s /var/lib/mysql/mysql.sock mysql.sock ls: cannot access /var/lib/mysql/mysql.sock: No such file or directory 0 mysql.sock +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ g4t7474_[root@g4t7474 tmp]# service mysqld start MySQL Daemon failed to start. Starting mysqld: [FAILED] ... View more

Mysql server do not start : ambari-server start

by Explorer in Support Questions
‎09-12-2017 08:53 PM
‎09-12-2017 08:53 PM
We have been working our sandbox cluster and we are not able to start mysql server getting the below error: g4t7474_[root@g4t7474 tools]# service mysqld status mysqld is stopped g4t7474_[root@g4t7474 tools]# service mysqld start MySQL Daemon failed to start. Starting mysqld: [FAILED] The Service stopped suddenly and when we tried to start it again it failed several times. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Please find attached two of the tested "my.cnf" files and the logs. my.cnf -> this is the original that was working my.cnf.back -> tried to replicate a working one but didnt succeed. mycnfback.txt mycnf.txt varlogmysqldlog.txt ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Below the mysql.sock "location" g4t7474_[root@g4t7474 ~]# locate mysql.sock /var/lib/mysql/mysql.sock g4t7474_[root@g4t7474 ~]# ls -ld /var/lib/mysql/mysql.sock ls: cannot access /var/lib/mysql/mysql.sock: No such file or directory g4t7474_[root@g4t7474 ~]# ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎01-30-2018 09:04 PM
Community Statistics
Member Since ‎07-31-2017 08:47 PM
Last Visited ‎01-30-2018 09:04 PM
Posts 10