Created on 06-20-2021 06:56 PM - edited 09-16-2022 07:42 AM
I use CDH 6.3.2
use oozie submit spark job,I get a error
.................................. at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141) Caused by: org.apache.hadoop.security.KerberosAuthException: failure to login: for principal: jztwk javax.security.auth.login.LoginException: Unable to obtain password from user at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1992) at org.apache.hadoop.security.UserGroupInformation.getUGIFromTicketCache(UserGroupInformation.java:649) ........................................ at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:93) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104) at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:60) ... 16 more Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
in Fixed Issues in CDH 6.3.3
I find
SPARK-29082 - [CORE] Skip delegation token generation if no credentials are available
so How can I fix it,only update to CDH 6.3.3
Created 06-20-2021 11:21 PM
@ighack You can try below workaround.
Do any of the following:
Use yarn-client mode for the SparkAction
OR
Still use yarn-cluster mode, however, add the submitting user's keytab on to a secure HDFS location and rewrite the workflow as shown:
<file>hdfs://xxx/yyy.keytab#zzz.keytab</file>
<spark-opts>--keytab zzz.keytab --principal zzz@YOUR_REALM</spark-opts>