Reply
New Contributor
Posts: 2
Registered: ‎04-30-2014

Hadoop WebHDFS SPNEGO auth failure from remote machine in trusted AD realm

We have a fresh Cloudera 5 installation on CentOS 6.5 and are using WebHDFS to perform file operations from Windows via a C# application. This all works fine when auth is disabled.

 

I have enabled Kerberos authentication using a local KDC (HADOOP.PVT) with a one-way trust to our active directory domain (COMPANY.PVT). This all appears to function properly. Locally I am able to use kinit to obtain a TGT from either user@HADOOP.PVT or user@COMPANY.PVT and connect to the Hadoop services (hive, hdfs, or WebHDFS using curl -i --negotiate -u : ).

When I attempt to make a WebHDFS connection from a remote Windows I receive the following error:

 

HTTP ERROR 401

Problem accessing /webhdfs/v1/user/. Reason:

org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Specified version of key is not available (44))

 

Since auth using an AD account works locally I'm at a loss as to which kvno it's complaining about.

We are using AES-256 and I have installed the appropriate JCE policy files.

 

Cloudera Employee
Posts: 229
Registered: ‎09-23-2013

Re: Hadoop WebHDFS SPNEGO auth failure from remote machine in trusted AD realm

the ksetup /addkdc command only modifies the registry of the system its run on, unlike the other commands for establishing trust, you must run it on systems that need to understand the realm to kdc mapping for the MIT kdc.  Any AD KDC referenced in the cluster's krb5.conf should have the command run on it.

 

Given what you are doing (windows app accessing secure cluster), you might also have to use ksetup /addhosttorealmmap to specificly map cluster hosts to the REALM that is configured by ksetup /addkdc.

 

The following links provide specific documentaiton on the two commands.  There is a way to set an entire subdomain as the target of the hosttorealmmap as well covered in the MS doc link, below.  Note that /addkdc requires a reboot to be set (addhosttorealmmap is ambigeous but I'm assuming its a good idea to set both values and then reboot..

 

ksetup /addkdc

http://technet.microsoft.com/en-us/library/hh240197.aspx

 

ksetup /addhosttorealmmap

http://technet.microsoft.com/en-us/library/hh240187.aspx

New Contributor
Posts: 2
Registered: ‎04-30-2014

Re: Hadoop WebHDFS SPNEGO auth failure from remote machine in trusted AD realm

Thank you for your reply. I tried following your steps and there was no change. For the purposes of our small development environment I have abandoned the MIT KDC and configured the cluster to directly use the AD servers as the KDC. After a little trial and error this is functioning perfectly.