Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Submit spark job via livy2 to HDP cluster with AD and Kerboros enabled

Highlighted

Submit spark job via livy2 to HDP cluster with AD and Kerboros enabled

New Contributor

I have a cluster running on HDP 2.6.5, and it has AD and Kerberos setup.

I am trying to spark job using livy following the steps below:-


1.On a MAC, set up krb5.conf.

[libdefaults]
  default_realm = ABC.DEF
 
[realms]
DATALAKE.LOCAL = {
   kdc = server.ABC.DEF
}

2. kinit username@ABC.DEF

3. klist

Credentials cache: API:DC55A167-B14F-4C45-8185-5B39C6C84366
        Principal: username@ABC.DEF

  Issued                Expires               Principal
Aug  2 11:26:10 2019  Aug  2 21:26:04 2019  krbtgt/ABC.DEF@ABC.DEF

4.

curl -v -ik --negotiate -u : --location-trusted -H 'Content-Type: application/json' http://xxx.xxx.xxx.xxx:port/sessions. Then I get this error

<h2>HTTP ERROR: 401</h2>
<p>Problem accessing /sessions. Reason:
<pre>    Authentication required</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>

Do you guys have any idea?