Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

401 Unauthorized Error Livy+ Hue

SOLVED Go to solution

401 Unauthorized Error Livy+ Hue

New Contributor

Hi Team, I recently configured Hue 3.11 with all backend configuration for Hive / WebHDFS etc. However while configuring Spaky/ Livy connectivity on Hue, I am getting 401 unauthorized under "Notebook" section on Hue and also while fetching session data through curl command. Below is the error that I am getting on command line: # curl  http://localhost:8998/sessions <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1" /> Error 401 HTTP ERROR: 401

Problem accessing /sessions. Reason:

    Authentication required

Powered by Jetty:// Below is the error from Hue: <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1" /> Error 401 HTTP ERROR: 401

Problem accessing /sessions. Reason:

 Authentication required

Powered by Jetty:// (error 401) Note : Cluster is kerberized.            Livy is configured with spnego kerberos principal /keytab as well as livy kerberos princial /keytab           Can someone help on this ? Regards Ajay Rathod

2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: 401 Unauthorized Error Livy Hue

Could you confirm on http://groups.google.com/a/cloudera.org/group/livy-user
if Livy supports Kerberos?

Re: 401 Unauthorized Error Livy Hue

8 REPLIES 8

Re: 401 Unauthorized Error Livy Hue

Could you confirm on http://groups.google.com/a/cloudera.org/group/livy-user
if Livy supports Kerberos?

Re: 401 Unauthorized Error Livy Hue

Re: 401 Unauthorized Error Livy Hue

New Contributor
Thanks Romain,

This helped me in fixing the issue.

Regards
Ajay Rathod

Re: 401 Unauthorized Error Livy Hue

Explorer
Hi @Romainr, in which section/configuration file I should set this parameter ( self._client.set_kerberos_auth() ) into Hue Service? Many thanks! Alex

Re: 401 Unauthorized Error Livy Hue

Super Guru

@AlexPQ,

 

Based on the referenced Jira, kerberos auth is now configurable.

 

In CDH 5.11.0 and higher, you can enable kerberos auth in hue.ini:

 

[spark]

security_enabled=true

 

 

Re: 401 Unauthorized Error Livy Hue

Explorer
Hi @bgooley,

Thanks a lot for the information!

Regards,

Alex

Re: 401 Unauthorized Error Livy Hue

Expert Contributor

The parameter:
[spark]
security_enabled=true

 

need to be added in Cloudera Manager Hue configuration to the below section:
Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini

 

However it still gave an error: Unable to authenticate . The Hue Admin screen showed a misconfiguration in Spark: The app wont work without a running Livy Spark Server. 

Also in  /var/log/hue/error.log below message was present:

GSSError: (('Unspecified GSS failure.  Minor code may provide more information', 851968), ('Server HTTP/localhost@EXAMPLE.COM not found in Kerberos database', -1765328377))

 

These errors were resolved by adding the line to the to the Hue Safety valve above:

[spark]

security_enabled=true

livy_server_host=yourlivyhostfqdn

 

Highlighted

Re: 401 Unauthorized Error Livy Hue

New Contributor

Dear Romainr:

    I follow your advise to modify the srcCode and made the hue3.9 notebook worked on my CDH5.7.1 Kerberized cluster with livy0.5.0.Now i can using spark-shell on notebook and its run well.

    but on yarn i saw the spark job user was always livy(i set livy be a haddop.proxyuser), seems its bind on the user which the keytab of livy-server-luancher. so i can't control the notebook  authority by hue.

    i can see the hue set proxyuser on a the token, but the spark job's user was not be reset.

 

 

livy-conf

livy.impersonation.enabled=true
livy.repl.enable-hive-context=true
livy.spark.deploy-mode=client
livy.spark.master=yarn
livy.superusers=hue
livy.server.auth.type=kerberos
livy.server.auth.kerberos.keytab=/etc/security/keytabs/spnego.keytab
livy.server.auth.kerberos.principal=HTTP/xxx.com@xxx.COM
livy.server.launch.kerberos.keytab=/etc/security/keytabs/livy.keytab
livy.server.launch.kerberos.principal=livy/xxx.com@xxx.COM

livy-log

19/04/12 14:48:14 INFO InteractiveSession$: Creating Interactive session 3: [owner: hue, request: [kind: pyspark, proxyUser: Some(baoyong), heartbeatTimeoutInSecond: 0]]
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 client token: Token { kind: YARN_CLIENT_TOKEN, service:  }
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 diagnostics: N/A
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 ApplicationMaster host: 192.168.103.166
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 ApplicationMaster RPC port: 0
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 queue: root.livy
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 start time: 1555051701595
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 final status: UNDEFINED
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 tracking URL: http://bigdata166.xxx.com:8088/proxy/application_1555044848792_0063/
19/04/12 14:48:25 INFO LineBufferedStream: stdout: 	 user: livy
19/04/12 14:48:28 INFO InteractiveSession: Interactive session 3 created [appid: application_1555044848792_0063, owner: hue, proxyUser: None, state: idle, kind: pyspark, info: {driverLogUrl=null, sparkUiUrl=null}]