Spark History Server failed to start

[#1]

Hi,

    I am using CDH 5.12 . I have installed using path A. Spark History server is not getting started. All, I am getting below logs continiously and history server is not starting.

2017-07-15 13:52:47,000 INFO org.apache.spark.SecurityManager: Changing view acls to: cloudera-scm
2017-07-15 13:52:47,001 INFO org.apache.spark.SecurityManager: Changing modify acls to: cloudera-scm
2017-07-15 13:52:47,001 INFO org.apache.spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cloudera-scm); users with modify permissions: Set(cloudera-scm)
2017-07-15 13:52:47,054 INFO org.apache.spark.deploy.history.FsHistoryProvider: History server ui acls disabled; users with admin permissions:
2017-07-15 13:52:47,060 WARN org.apache.spark.SparkConf: The configuration key 'spark.history.fs.update.interval.seconds' has been deprecated as of Spark 1.4 and and may be removed in the future. Please use the new key 'spark.history.fs.update.interval' instead.
2017-07-15 13:52:58,260 INFO org.apache.spark.deploy.history.HistoryServer: Started daemon with process name: 2561@compute-15
2017-07-15 13:52:58,273 INFO org.apache.spark.deploy.history.HistoryServer: Registered signal handlers for [TERM, HUP, INT]
2017-07-15 13:52:58,312 WARN org.apache.spark.SparkConf: The configuration key 'spark.history.fs.update.interval.seconds' has been deprecated as of Spark 1.4 and and may be removed in the future. Please use the new key 'spark.history.fs.update.interval' instead.

Is there something wrong in configuration ? How can this be solved ?

Thanks.

[#2]

@vikash145

According to this log, your user id doesn't have permission to start the server, login as root and try again

additionally you can check the 'service <service name> status>' and try to start it

also check the config by running the below command and on it

chkconfig --list

[#3]

@saranvisa

The user is cloudera-scm. All the other services are running fine. Are u suggesting apart from sudoers passwordless permission, I should set any other permission ? Please let me know , what and where this permission need to be set.

Thanks

[#4]

@vikash145

when do you get this error? when you try to launch spark shell (or) when you start the history servicer (or) in a different situation?

how did you start all other services? using the same user

also i don't see any keyword called error in your log, it says only warning

[#5]

@mbigelow

Any pointer on this ?

[#6]

There isn't anything of use in the logs you posted. Are you able to post the full log?

Also check the stderr log, either in CM or in the process running directory, as I think this is where it posted launch errors.

[#7]

Once this HIVE service is launched. I will post the log.

[#8 · Accepted solution]

/user/spark directory was missing. I created it manually and it worked fine.