question Re: Why does Ambari use headless keytab for Spark history server in Archives of Support Questions (Read Only)

Why does Ambari use headless keytab for Spark history server

vjiang — Tue, 12 Jan 2016 06:37:38 GMT

It looks like Ambari uses headless keytab for Spark history server:

Execute['/usr/bin/kinit -kt /dsap/etc/security/keytabs/spark.headless.keytab spark-abc@EXAMPLE.COM; '] {'user': 'spark'}

Does anyone know why?

Also current documentation suggests that we need to create keytabs on host as described here:

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_spark-guide/content/ch_installing-kerb-spark.html

This is somehow confusing.

Re: Why does Ambari use headless keytab for Spark history server

vshukla — Tue, 12 Jan 2016 06:41:36 GMT

Ambari needs headless Keytab so it can start services without prompting for password. That's how Spark's History server is started (i.e. w/o prompting for password)

When an end user is submitting a spark job, they can use either a headless keytab or type the kerberos password.

Re: Why does Ambari use headless keytab for Spark history server

rlevas — Tue, 12 Jan 2016 07:54:27 GMT

@Vincent Jiang, What is confusing... the fact that a keytab is being used or that a headless principal is being used?

Re: Why does Ambari use headless keytab for Spark history server

hosako — Thu, 14 Jan 2016 07:28:50 GMT

Is there any possibility that this headless keytab is used when spark submits a job (to YARN or hive, maybe?) to identify itself?

Not for ambari to start Spark service, maybe?

Re: Why does Ambari use headless keytab for Spark history server

stevel — Fri, 15 Jan 2016 01:52:20 GMT

Afraid not. The same keytab could be used if you had a local copy of it when you submitted work. Otherwise, when you submit a Spark job to the YARN cluster, it picks up your credentials, grabbing a Hive and HBase token if needed, and uses them for the duration of the job.

Note that because those tokens expire after a day or two, you can't do long-lived applications that way. You will need a keytab, and spark 1.5, which is where keytab-based Spark application support went in,