Support Questions
Find answers, ask questions, and share your expertise

Connecting to livy in kerberized cluster

Hi all,

My Livy service (version 0.3) is configured as shown in this tutorial:

http://henning.kropponline.de/2016/11/06/connecting-livy-to-a-secured-kerberized-hdp-cluster/

The following command is able to access the livy server and give me a result:

curl --negotiate -u : http://fqdn.host.name:8998/sessions

My problem is when I try to launch a job from my API, I get an authentication error. Is this a problem due to configurations or because my code isn't asking for any negotiation methods/isn't using principals ?

Before running the jobs I have tried doing a 'kinit' for SPNEGO, Zeppelin and Livy principals..no luck..

4 REPLIES 4

@L V,

I'm not sure in which language you are writing the code. I used python and it works for me. Do a kinit before running the script and run the script

import requests
from requests_kerberos import HTTPKerberosAuth, REQUIRED
import json
session_url = "http://fqdn.host.name:8998/sessions"
kerberos_auth = HTTPKerberosAuth(mutual_authentication=REQUIRED, sanitize_mutual_error_response=False)
headers = { 'X-Requested-By': 'livy'}
response = requests.get(session_url, headers=headers, auth=kerberos_auth, verify=False)
print response.status_code
print response.text

Install the kerberos package before running the script

pip install requests-kerberos

Thanks,

Aditya

New Contributor

@Aditya Sirna

Hi Aditya I am able to execute the above code but I am getting issue while running post method. can you help me out on this.

I have done kinit before running below commands.

import json, pprint, requests, textwrap

from requests_kerberos import HTTPKerberosAuth, REQUIRED

session_url = "http://localhost:8998/sessions"

kerberos_auth = HTTPKerberosAuth(mutual_authentication=REQUIRED, sanitize_mutual_error_response=False)

headers = { 'X-Requested-By': 'livy'}

data = {'kind': 'spark'}

response = requests.post(session_url, headers=headers, data=json.dumps(data), auth=kerberos_auth, verify=False)

print response.status_code

response.json()

@Venkatesh Madala,

What is the error that you are facing?

New Contributor

@Aditya Sirna

Below is the Error I am getting. response code is 401

>>> print response.status_code

401

>>> response.json()

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json

return json.loads(self.text, **kwargs)

File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads

return _default_decoder.decode(s)

File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode

obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode

raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded

>>>

; ;