On the host that is showing this error, can you klist the following keytab files: /etc/security/keytabs/spnego.service.keytab /etc/security/keytabs/dn.service.keytab [root@c6501 ~]# klist -kte /etc/security/keytabs/spnego.service.keytab Keytab name: FILE:/etc/security/keytabs/spnego.service.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 12/11/15 15:38:21 HTTP/c6501.ambari.apache.org@EXAMPLE.COM (des-cbc-md5) 1 12/11/15 15:38:21 HTTP/c6501.ambari.apache.org@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 1 12/11/15 15:38:21 HTTP/c6501.ambari.apache.org@EXAMPLE.COM (des3-cbc-sha1) 1 12/11/15 15:38:21 HTTP/c6501.ambari.apache.org@EXAMPLE.COM (arcfour-hmac) 1 12/11/15 15:38:21 HTTP/c6501.ambari.apache.org@EXAMPLE.COM (aes128-cts-hmac-sha1-96) [root@c6501 ~]# klist -kte /etc/security/keytabs/dn.service.keytab Keytab name: FILE:/etc/security/keytabs/dn.service.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 12/11/15 15:38:21 dn/c6501.ambari.apache.org@EXAMPLE.COM (des-cbc-md5) 1 12/11/15 15:38:21 dn/c6501.ambari.apache.org@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 1 12/11/15 15:38:21 dn/c6501.ambari.apache.org@EXAMPLE.COM (des3-cbc-sha1) 1 12/11/15 15:38:21 dn/c6501.ambari.apache.org@EXAMPLE.COM (arcfour-hmac) 1 12/11/15 15:38:21 dn/c6501.ambari.apache.org@EXAMPLE.COM (aes128-cts-hmac-sha1-96) Notice that my examples have entries for aes256-cts-hmac-sha1-96, do yours? If not, then you will need to make sure that you have the Unlimited Key JCE policy jars (for your JVM) installed. If these look good, then from the host you having the issue with, can you attempt to kinit using them. If you get no messages, all is good... else there is a disconnect between your keytab file and the password stored in the KDC. Success case: [root@c6501 ~]# kinit -kt /etc/security/keytabs/dn.service.keytab dn/c6501.ambari.apache.org@EXAMPLE.COM [root@c6501 ~]# Failure case: [root@c6501 ~]# kinit -kt /etc/security/keytabs/dn.service.keytab dn/c6501.ambari.apache.org@EXAMPLE.COM kinit: Password incorrect while getting initial credentials [root@c6501 ~]# If you get this failure, then try to regenerate the keytab files from the Ambari UI to see if that helps the issue. ... View more

This is an authentication error that occurs when an HTTP request lacks a Kerberos ticket while the server requires it. So some service is not properly configured or a kinit is failing prior to executing this webhdfs call. The service that is failing needs to be identified. Then you need to take a look at how it is configured and make sure a kinit can be successfully executed (manually) using the relevant Kerberos principal and keytab file. The kinit may fail because the keytab file is out of sync with the KDC or the service is misconfigured to use an incorrect principal name or keytab file. ... View more

You are using the wrong principal name for that keytab file. To see what keytab entries in that keytab file, use klist klist -kte /etc/security/keytabs/hdfs.headless.keytab This will list out the contents of the keytab file. There will be 1 or more lines indicating details about each keytab entry. The 2nd-to-last column will show the principal name for each entry. The last column will show the encryption algorithm used to create the keytab entry. ... View more

The HTTP 302 code is a redirect response. This shouldn't affect your ability to authenticate with the web app via your browser. So something with your configuration may be wrong. I assume that you entered the Oozie UI server information in as a trusted URI. Also, maybe this might help - http://www.microhowto.info/howto/configure_firefox_to_authenticate_using_spnego_and_kerberos.html ... View more

Once the account has been created, you can use the "Active Directory Users and Computers" applet on the Windows Server, find the account, open up the properties, and perform one of the following: Option A: Click on the "Account" tab Edit the 2nd box of the "Windows login name (pre-Windows 2000)" field Click the "Apply" button Option B: Click on the "Attribute Editor" tab (shown if "Advanced Features" is turned on) Look for "sAMAccountName" in the listbox Double click on the "sAMAccountName" item - a small dialog box will pop up Edit the name in the textbox Click the "OK" button (closed the dialog box) Click the "Apply" button If you haven't enabled Kerberos yet, you can edit the Active Directory "Account Attribute Template" value under the "Advanced kerberos-env" section on the "Configure Kerberos" page of the "Enable Kerberos Wizard" and add something like: "sAMAccountName": "$principal_name" However this field is limited to 20 characters so you will need to do some additional stuff to limit the value and hope that it is unique. For example: #set( $sAMAccountName = "${principal_primary}-${principal_digest}" ) { "objectClass": ["top","person","organizationalPerson","user"], "cn": "$principal_name", "sAMAccountName": "$sAMAccountName.substring(0,20)", #if( $is_service ) "servicePrincipalName": "$principal_name", #end "userPrincipalName": "$normalized_principal", "unicodePwd": "$password", "accountExpires": "0", "userAccountControl": "66048" } ... View more

When Ambari creates passwords for accounts it generates a string of 20 characters chosen from a set of various character classes - uppercase characters, lowercase characters, digits, punctuation, and whitespace. A minimum number of instances of each character class can be configured to help meet password policies that might be applied to the KDC. ... View more

Assuming this is for Ambari 2.x, when Ambari is managing Kerberos identities it is expected that each identity is unique across all clusters that share a KDC or AD. By default, Ambari provides this appending the cluster name (ideally unique across clusters sharing the same KDC or AD) to the headless principals it manages. For example the HDFS Kerberos identity typically has a principal name of hdfs-${CLUSTER_NAME}@${REALM}. So if the realm was AD.EXAMPLE.COM and there were 2 clusters that share the AD - Cluster1 and Cluster2 - then Ambari will create the following accounts in the AD: hdfs-Cluster1@AD.EXAMPLE.COM and hdfs-Cluster2@AD.EXAMPLE.COM. This is needed so that Ambari does not change the password for one cluster's identities while performing Kerberos-related tasks for another cluster. Ideally each cluster points to a different container (or OU), but that is for convenience. This doesn't make a difference for Ambari. When it comes to service identities, Ambari relies on the uniqueness of the hostnames to create the service principals. This is because service principals have a hostname associated with them. For example, nn/h1c1.example.com@AD.EXAMPLE.COM. For non-managed (user) identities, none of this makes a difference. Since Ambari does not manage these, Ambari will not change the accounts in anyway. So they are safe to manage manually and use across clusters. Note: The Ambari-managed AD accounts need to be left alone by administrators. Since changing them may prevent Ambari or the related hadoop services from authenticating properly. ... View more

This appears to look correct. Are we sure the realm name is correct and it is not something like "TCORP.COM"? Realm names are case-sensitive, so make sure the realm name in AD is all uppercase characters. I don't believe that the admin principal or password is trimmed, so make sure no (extra) spaces exist before or after them. Also, does the admin user have delegated control over the specified LDAP container? Can you take a look at the Ambari server log to see if any errors are posted there? ... View more