Spark-submit - mapping of principal

[#1]

Hello,
on our system CDP 7.1.7, we have use konfiguration parameter
kafka.properties_role_safety_valve
add set attribute
sasl.kerberos.principal.to.local.rules
to map ActiveDirectory principals to entities created in ranger.

In our system, the AD user have a prefix e.g. [email protected] maps to a ranger entity "johndoe"

During a spark-submit (over yarn), we also need to pass a principal, however as there is no such mapping, we obtain an error saying the unix user "xjohndoe" does not exist. This is true indeed, we eed to map it to "johndoe".

Ist there any possibility to map principals during spark-spark-submit possibly similarly to sasl.kerberos.principal.to.local.rules in kafka or any other possibility?

Best regards
Jaro

[#2 · Accepted solution]

Hi @Jarinek , Yes, in CDH/CDP every service which depends on HDFS will inherit the HDFS configuration "auth-to-local rules", in CM in HDFS Configuration see "Additional Rules to Map Kerberos Principals to Short Names".

Kafka does not need HDFS so that's why it has a separate such configuration.

See the documentation how to set it:

https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/security-kerberos-authentication/topics/cm-se...

Best regards

 Miklos

[#3]

Great, thanks