Reply
Explorer
Posts: 31
Registered: ‎07-11-2017
Accepted Solution

Spark History Server failed to start

[ Edited ]

Hi,

 

    I am using CDH 5.12 . I have installed using path A. Spark History server is not getting started. All, I am getting below logs continiously and history server is not starting.

 

2017-07-15 13:52:47,000 INFO org.apache.spark.SecurityManager: Changing view acls to: cloudera-scm
2017-07-15 13:52:47,001 INFO org.apache.spark.SecurityManager: Changing modify acls to: cloudera-scm
2017-07-15 13:52:47,001 INFO org.apache.spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cloudera-scm); users with modify permissions: Set(cloudera-scm)
2017-07-15 13:52:47,054 INFO org.apache.spark.deploy.history.FsHistoryProvider: History server ui acls disabled; users with admin permissions:
2017-07-15 13:52:47,060 WARN org.apache.spark.SparkConf: The configuration key 'spark.history.fs.update.interval.seconds' has been deprecated as of Spark 1.4 and and may be removed in the future. Please use the new key 'spark.history.fs.update.interval' instead.
2017-07-15 13:52:58,260 INFO org.apache.spark.deploy.history.HistoryServer: Started daemon with process name: 2561@compute-15
2017-07-15 13:52:58,273 INFO org.apache.spark.deploy.history.HistoryServer: Registered signal handlers for [TERM, HUP, INT]
2017-07-15 13:52:58,312 WARN org.apache.spark.SparkConf: The configuration key 'spark.history.fs.update.interval.seconds' has been deprecated as of Spark 1.4 and and may be removed in the future. Please use the new key 'spark.history.fs.update.interval' instead.

 

 

Is there something wrong in configuration ? How can this be solved ?

 

Thanks.

Posts: 365
Topics: 11
Kudos: 57
Solutions: 30
Registered: ‎09-02-2016

Re: Spark History Server failed to start

@vikash145

 

According to this log, your user id doesn't have permission to start the server, login as root and try again

 

additionally you can check the 'service <service name> status>' and try to start it

also check the config by running the below command and on it

chkconfig --list

 

Explorer
Posts: 31
Registered: ‎07-11-2017

Re: Spark History Server failed to start

@saranvisa

 

The user is cloudera-scm. All the other services are running fine. Are u suggesting apart from sudoers passwordless permission, I should set any other permission ? Please let me know , what and where this permission need to be set.

 

 

Thanks

Highlighted
Posts: 365
Topics: 11
Kudos: 57
Solutions: 30
Registered: ‎09-02-2016

Re: Spark History Server failed to start

@vikash145

 

when do you get this error? when you try to launch spark shell (or) when you start the history servicer (or) in a different situation?

 

how did you start all other services? using the same user

 

also i don't see any keyword called error in your log, it says only warning

Explorer
Posts: 31
Registered: ‎07-11-2017

Re: Spark History Server failed to start

@mbigelow

Any pointer on this ?
Posts: 630
Topics: 3
Kudos: 102
Solutions: 66
Registered: ‎08-16-2016

Re: Spark History Server failed to start

There isn't anything of use in the logs you posted. Are you able to post the full log?

Also check the stderr log, either in CM or in the process running directory, as I think this is where it posted launch errors.
Explorer
Posts: 31
Registered: ‎07-11-2017

Re: Spark History Server failed to start

Once this HIVE service is launched. I will post the log.

Explorer
Posts: 31
Registered: ‎07-11-2017

Re: Spark History Server failed to start

[ Edited ]

/user/spark directory was missing. I created it manually and it worked fine.

Announcements