Spark History Server kerberos error

[#1]

Hi,

Since kerberos is activated, spark history server doesn't start.

9:33:47.771 AM	WARN	org.apache.hadoop.ipc.Client	Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate ailed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
9:33:47.771 AM	WARN	org.apache.hadoop.security.UserGroupInformation	PriviledgedActionException as:spark (auth:KERBEROS) cause:java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

All other services are OK.

Someone has already had the same error ?

[#2]

Hi,

Are you using spark on yarn or standalone spark? Note that standalone spark doesn't work with Kerberos.

Which version of CM are you using? Which version of CDH?

Thanks,
Darren

[#3]

Hi,

CDH 5.1.2 is used, so we have only one version of spark.

regards,

[#4]

Hi,

Your CDH version is 5.1.2. What is the Cloudera Manager version? Note that these can be different numbers.

Cloudera Manager allows you to add either Spark on YARN and Standalone Spark. These aren't different versions of the Spark binaries, they're different ways to deploy spark. In Cloudera Manager, these two are different service types. Which service type did you install? If you aren't sure which you are using, look at the Instances tab for your spark service. If you have any Master or Worker roles, then you're using Standalone Spark.

Are you using Spark on YARN or Standalone Spark?

Thanks,
Darren

[#5]

Hi Darren,

thank you for your reply.

My CM is 5.1.2 and i'm using Standalone spark.

regards,

[#6]

petibonvm,

I also had this issue in CDH 5.1.x.  My advice would be to upgrade to CM/CDH 5.3 and use Spark on YARN.  If that is not currently possible, the following work around is posted in the CM 5.1.x Release Notes:

The Spark History Server does not start when Kerberos authentication is enabled.

The Spark History Server does not start when managed by a Cloudera Manager 5.1 instance when Kerberos authentication is enabled.

Workaround:

1. Go to the Spark service.
2. Expand the History Server Default Group > Advanced category.
3. Add the following configuration to the History Server Environment Advanced Configuration Snippet property:

   SPARK_HISTORY_OPTS=-Dspark.history.kerberos.enabled=true \
   -Dspark.history.kerberos.principal=principal \
   -Dspark.history.kerberos.keytab=keytab

where principal is the name of the Kerberos principal to use for the History Server, and keytab is the path to the principal's keytab file on the local filesystem of the host running the History Server.

Regards,

Chad

[#7 · Accepted solution]

Hi,

thank you for your feedback.

I have upgraded into CDH 5.3 and spark history server (via yarn) was successfully started.

regards,