Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark History Server failed to start

avatar
Contributor

Hi,

 

    I am using CDH 5.12 . I have installed using path A. Spark History server is not getting started. All, I am getting below logs continiously and history server is not starting.

 

2017-07-15 13:52:47,000 INFO org.apache.spark.SecurityManager: Changing view acls to: cloudera-scm
2017-07-15 13:52:47,001 INFO org.apache.spark.SecurityManager: Changing modify acls to: cloudera-scm
2017-07-15 13:52:47,001 INFO org.apache.spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cloudera-scm); users with modify permissions: Set(cloudera-scm)
2017-07-15 13:52:47,054 INFO org.apache.spark.deploy.history.FsHistoryProvider: History server ui acls disabled; users with admin permissions:
2017-07-15 13:52:47,060 WARN org.apache.spark.SparkConf: The configuration key 'spark.history.fs.update.interval.seconds' has been deprecated as of Spark 1.4 and and may be removed in the future. Please use the new key 'spark.history.fs.update.interval' instead.
2017-07-15 13:52:58,260 INFO org.apache.spark.deploy.history.HistoryServer: Started daemon with process name: 2561@compute-15
2017-07-15 13:52:58,273 INFO org.apache.spark.deploy.history.HistoryServer: Registered signal handlers for [TERM, HUP, INT]
2017-07-15 13:52:58,312 WARN org.apache.spark.SparkConf: The configuration key 'spark.history.fs.update.interval.seconds' has been deprecated as of Spark 1.4 and and may be removed in the future. Please use the new key 'spark.history.fs.update.interval' instead.

 

 

Is there something wrong in configuration ? How can this be solved ?

 

Thanks.

1 ACCEPTED SOLUTION

avatar
Contributor

/user/spark directory was missing. I created it manually and it worked fine.

View solution in original post

7 REPLIES 7

avatar
Champion

@vikash145

 

According to this log, your user id doesn't have permission to start the server, login as root and try again

 

additionally you can check the 'service <service name> status>' and try to start it

also check the config by running the below command and on it

chkconfig --list

 

avatar
Contributor

@saranvisa

 

The user is cloudera-scm. All the other services are running fine. Are u suggesting apart from sudoers passwordless permission, I should set any other permission ? Please let me know , what and where this permission need to be set.

 

 

Thanks

avatar
Champion

@vikash145

 

when do you get this error? when you try to launch spark shell (or) when you start the history servicer (or) in a different situation?

 

how did you start all other services? using the same user

 

also i don't see any keyword called error in your log, it says only warning

avatar
Contributor
@mbigelow

Any pointer on this ?

avatar
Champion
There isn't anything of use in the logs you posted. Are you able to post the full log?

Also check the stderr log, either in CM or in the process running directory, as I think this is where it posted launch errors.

avatar
Contributor

Once this HIVE service is launched. I will post the log.

avatar
Contributor

/user/spark directory was missing. I created it manually and it worked fine.