ambari Views fail after Kerberos Enabled.

[#1]

Hi,

I am not able to view any ambari views except Yarn after kerberos enabled. I dont have any proxy users setup and just have ambari server.

Any suggestions Please :

How to configure after kerberos enabled:

Hive View :

Issues detected
Service 'ats' check failed: Server ErrorService 'userhome' check failed: Authentication required

Service 'userhome' check failed:
org.apache.hadoop.security.AccessControlException: Authentication required
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:457)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:113)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:738)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:582)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:612)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:608)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHdfsFileStatus(WebHdfsFileSystem.java:987)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getFileStatus(WebHdfsFileSystem.java:1003)
	at org.apache.ambari.view.utils.hdfs.HdfsApi$3.run(HdfsApi.java:127)
	at org.apache.ambari.view.utils.hdfs.HdfsApi$3.run(HdfsApi.java:125)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)

Trace : Ambari Files View

Authentication required

org.apache.hadoop.security.AccessControlException: Authentication required
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:457)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:113)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:738)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:582)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:612)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)

[#2]

@Sam Red

Here we are talking of the user running the ambari processes check like below

# ls -al /etc/ambari-server/conf/

Regards

[#3]

@Geoffrey Shelton Okot

total 28
drwxr-xr-x 2 root root  131 Aug 29 11:49 .
drwxr-xr-x 3 root root   18 Aug  1 22:37 ..
-rwxrwxrwx 1 root root 6824 Aug 24 13:02 ambari.properties
-rwxrwxrwx 1 root root  311 Aug 29 11:49 krb5JAASLogin.conf
-rw-r--r-- 1 root root  286 Aug 29 11:49 krb5JAASLogin.conf.bak
-rwxrwxrwx 1 root root 4929 Aug  1 22:37 log4j.properties
-rw-r----- 1 root root    7 Aug  1 22:47 password.dat

[#4 · Accepted solution]

@Sam Red

Then you have to use

hadoop.proxyuser.root.groups=*
hadoop.proxyuser.root.hosts=* 

hadoop.proxyuser.ambari-server.groups=*
hadoop.proxyuser.ambari-server.hosts=*

[#5]

@Geoffrey Shelton Okot

[#6]

@Sam Red

The bold part obscured above should be part Authentication part in the File/hive view

hadoop.proxyuser.ambari-server-xxxx.hosts

hadoop.proxyuser.ambari-server-xxxx.groups

In the part of the views

WebHDFS Authentication : auth=KERBEROS;proxyuser=ambari-server-xxxx@REALM

[#7]

@Geoffrey Shelton Okot

After restartred ambari server got new issue :

Service 'hdfs' check failed:
java.lang.NullPointerException
	at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:383)
	at org.apache.hadoop.security.User.<init>(User.java:48)
	at org.apache.hadoop.security.User.<init>(User.java:43)
	at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1270)
	at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1254)
	at org.apache.ambari.view.utils.hdfs.HdfsApi.getProxyUser(HdfsApi.java:78)
	at org.apache.ambari.view.utils.hdfs.HdfsApi.<init>(HdfsApi.java:66)
	at org.apache.ambari.view.utils.hdfs.HdfsUtil.connectToHDFSApi(HdfsUtil.java:127)
	at org.apache.ambari.view.commons.hdfs.HdfsService.hdfsSmokeTest(HdfsService.java:136)
	at org.apache.ambari.view.filebrowser.HelpService.hdfsStatus(HelpService.java:86)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)

[#8]

@Sam Red

2 things to do can you first restart the cluster and see if it persists.

What is the value for WebHDFS Authentication in the views ?

From the stack trace the problem is due to the mapping from full Kerberos principal name to short username. This mapping is driven by the following configuration property in core-site.xml.

<property>
  <name>hadoop.security.auth_to_local</name>
  <value></value>
  <description>Maps kerberos principals to local user names</description>
</property>

Please revert

[#9]

@Geoffrey Shelton Okot

I don't know what is the issue. did lot of research but still issue persist.

Webhdfs : auth=KERBEROS;proxyuser=ambari-server-abc_bigpipeline@RELAY.COM

[#10]

@Sam Red

I have also been trying to understand what is wrong. What's this command's output?

# klist -kt /etc/security/keytabs/ambari.server.keytab
keytab name: FILE:/etc/security/keytabs/ambari.server.keytab
KVNO           Timestamp           Principal
---- ------------------- ------------------------------------------------------
   1 08/24/2017 15:42:24 ambari-server-abc_bigxxxline@ROMAT.COM
   1 08/24/2017 15:42:24 ambari-server-abc_bigxxxline@ROMAT.COM
   1 08/24/2017 15:42:24 ambari-server-abc_bigxxxline@ROMAT.COM
   1 08/24/2017 15:42:24 ambari-server-abc_bigxxxline@ROMAT.COM
   1 08/24/2017 15:42:24 ambari-server-abc_bigxxxline@ROMAT.COM

Then grab a valid Kerberos ticket

$ kinit -kt /etc/security/keytabs/ambari.server.keytab   ambari-server-abc_bigxxxline@ROMAT.COM

Then try accessing then retry.

[#11]

@Geoffrey Shelton Okot

Thank You. after lot of edits i am able to open FilesView But not Hive View.

Issues detected
Service 'hdfs' check failed: E090 NullPointerException
Service 'userhome' check failed: HdfsApi connection failed. Check "webhdfs.url" property