Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Why does Ambari use headless keytab for Spark history server

avatar
Rising Star

It looks like Ambari uses headless keytab for Spark history server:

Execute['/usr/bin/kinit -kt /dsap/etc/security/keytabs/spark.headless.keytab spark-abc@EXAMPLE.COM; '] {'user': 'spark'}

Does anyone know why?

Also current documentation suggests that we need to create keytabs on host as described here:

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_spark-guide/content/ch_installing-kerb-sp...

This is somehow confusing.

1 ACCEPTED SOLUTION

avatar

Ambari needs headless Keytab so it can start services without prompting for password. That's how Spark's History server is started (i.e. w/o prompting for password)

When an end user is submitting a spark job, they can use either a headless keytab or type the kerberos password.

View solution in original post

4 REPLIES 4

avatar

Ambari needs headless Keytab so it can start services without prompting for password. That's how Spark's History server is started (i.e. w/o prompting for password)

When an end user is submitting a spark job, they can use either a headless keytab or type the kerberos password.

avatar

Is there any possibility that this headless keytab is used when spark submits a job (to YARN or hive, maybe?) to identify itself?

Not for ambari to start Spark service, maybe?

avatar

Afraid not. The same keytab could be used if you had a local copy of it when you submitted work. Otherwise, when you submit a Spark job to the YARN cluster, it picks up your credentials, grabbing a Hive and HBase token if needed, and uses them for the duration of the job.

Note that because those tokens expire after a day or two, you can't do long-lived applications that way. You will need a keytab, and spark 1.5, which is where keytab-based Spark application support went in,

avatar

@Vincent Jiang, What is confusing... the fact that a keytab is being used or that a headless principal is being used?