Support Questions
Find answers, ask questions, and share your expertise

Why does Ambari use headless keytab for Spark history server

Contributor

It looks like Ambari uses headless keytab for Spark history server:

Execute['/usr/bin/kinit -kt /dsap/etc/security/keytabs/spark.headless.keytab spark-abc@EXAMPLE.COM; '] {'user': 'spark'}

Does anyone know why?

Also current documentation suggests that we need to create keytabs on host as described here:

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_spark-guide/content/ch_installing-kerb-sp...

This is somehow confusing.

1 ACCEPTED SOLUTION

Accepted Solutions

Ambari needs headless Keytab so it can start services without prompting for password. That's how Spark's History server is started (i.e. w/o prompting for password)

When an end user is submitting a spark job, they can use either a headless keytab or type the kerberos password.

View solution in original post

4 REPLIES 4

Ambari needs headless Keytab so it can start services without prompting for password. That's how Spark's History server is started (i.e. w/o prompting for password)

When an end user is submitting a spark job, they can use either a headless keytab or type the kerberos password.

View solution in original post

Is there any possibility that this headless keytab is used when spark submits a job (to YARN or hive, maybe?) to identify itself?

Not for ambari to start Spark service, maybe?

Afraid not. The same keytab could be used if you had a local copy of it when you submitted work. Otherwise, when you submit a Spark job to the YARN cluster, it picks up your credentials, grabbing a Hive and HBase token if needed, and uses them for the duration of the job.

Note that because those tokens expire after a day or two, you can't do long-lived applications that way. You will need a keytab, and spark 1.5, which is where keytab-based Spark application support went in,

@Vincent Jiang, What is confusing... the fact that a keytab is being used or that a headless principal is being used?