Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Connecting to livy in kerberized cluster

Connecting to livy in kerberized cluster

New Contributor

Hi all,

My Livy service (version 0.3) is configured as shown in this tutorial:

http://henning.kropponline.de/2016/11/06/connecting-livy-to-a-secured-kerberized-hdp-cluster/

The following command is able to access the livy server and give me a result:

curl --negotiate -u : http://fqdn.host.name:8998/sessions

My problem is when I try to launch a job from my API, I get an authentication error. Is this a problem due to configurations or because my code isn't asking for any negotiation methods/isn't using principals ?

Before running the jobs I have tried doing a 'kinit' for SPNEGO, Zeppelin and Livy principals..no luck..

4 REPLIES 4

Re: Connecting to livy in kerberized cluster

@L V,

I'm not sure in which language you are writing the code. I used python and it works for me. Do a kinit before running the script and run the script

import requests
from requests_kerberos import HTTPKerberosAuth, REQUIRED
import json
session_url = "http://fqdn.host.name:8998/sessions"
kerberos_auth = HTTPKerberosAuth(mutual_authentication=REQUIRED, sanitize_mutual_error_response=False)
headers = { 'X-Requested-By': 'livy'}
response = requests.get(session_url, headers=headers, auth=kerberos_auth, verify=False)
print response.status_code
print response.text

Install the kerberos package before running the script

pip install requests-kerberos

Thanks,

Aditya

Re: Connecting to livy in kerberized cluster

New Contributor

@Aditya Sirna

Hi Aditya I am able to execute the above code but I am getting issue while running post method. can you help me out on this.

I have done kinit before running below commands.

import json, pprint, requests, textwrap

from requests_kerberos import HTTPKerberosAuth, REQUIRED

session_url = "http://localhost:8998/sessions"

kerberos_auth = HTTPKerberosAuth(mutual_authentication=REQUIRED, sanitize_mutual_error_response=False)

headers = { 'X-Requested-By': 'livy'}

data = {'kind': 'spark'}

response = requests.post(session_url, headers=headers, data=json.dumps(data), auth=kerberos_auth, verify=False)

print response.status_code

response.json()

Highlighted

Re: Connecting to livy in kerberized cluster

@Venkatesh Madala,

What is the error that you are facing?

Re: Connecting to livy in kerberized cluster

New Contributor

@Aditya Sirna

Below is the Error I am getting. response code is 401

>>> print response.status_code

401

>>> response.json()

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json

return json.loads(self.text, **kwargs)

File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads

return _default_decoder.decode(s)

File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode

obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode

raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded

>>>

Don't have an account?
Coming from Hortonworks? Activate your account here