Getting error when trying to Generate Missing Credentials

[#1]

When trying to Generate Missing Credentials in Cloudera Manager I'm getting an error.

I'm interfacing with Active Directory.

It seems to me that there is a bug in the /usr/share/cmf/gen_credentials_ad.sh script.  In the Security section Cloudera Manager reports that the status is "Successfully Enabled Kerberos" so it seems I'm connecting to the Domain/KDC just fine.

Anybody have these troubles before?  I'm using Cloudera Manager 5.8.2.  Red Hat 7.2.

/usr/share/cmf/bin/gen_credentials_ad.sh failed with exit code 16 and output of <<
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ KEYTAB_OUT=/var/run/cloudera-scm-server/cmf3470099377124057462.keytab
+ PRINC=hbase/pp-hadoop-sec1.accelrys.net@ACCELRYS.NET
+ USER=DGLtZkeNFI
+ PASSWD='REDACTED'
+ PASSWD='AaBb=REDACTED-'
+ DELETE_ON_REGENERATE=false
+ SET_ENCRYPTION_TYPES=true
+ ENC_TYPES_MASK=4
+ USERACCOUNTCONTROL=66048
+ ACCOUNTEXPIRES=0
+ OBJECTCLASSES='objectClass: top
'
+ DIST_NAME=CN=DGLtZkeNFI,ou=pp-hadoop,DC=accelrys,DC=net
+ '[' -z /var/run/cloudera-scm-server/krb53587799783946059745.conf ']'
+ echo 'Using custom config path '\''/var/run/cloudera-scm-server/krb53587799783946059745.conf'\'', contents below:'
+ cat /var/run/cloudera-scm-server/krb53587799783946059745.conf
+ SIMPLE_PWD_STR=
+ '[' '' = '' ']'
+ kinit -k -t /var/run/cloudera-scm-server/cmf3506971145990435068.keytab pp-hadoop@ACCELRYS.NET
++ mktemp /tmp/cm_ldap.XXXXXXXX
+ LDAP_CONF=/tmp/cm_ldap.jT0pFrVm
+ echo 'TLS_REQCERT     never'
+ echo 'sasl_secprops   minssf=0,maxssf=0'
+ export LDAPCONF=/tmp/cm_ldap.jT0pFrVm
+ LDAPCONF=/tmp/cm_ldap.jT0pFrVm
++ ldapsearch -LLL -H ldaps://dc1-accl-uswst.accelrys.net:636 -b ou=pp-hadoop,DC=accelrys,DC=net userPrincipalName=hbase/pp-hadoop-sec1.accelrys.net@ACCELRYS.NET
SASL/GSSAPI authentication started
SASL username: pp-hadoop@ACCELRYS.NET
SASL SSF: 0
+ PRINC_SEARCH=
+ set +e
+ echo
+ grep -q userPrincipalName
+ '[' 1 -eq 0 ']'
+ set -e
+ '[' true = true ']'
+ ldapmodify -H ldaps://dc1-accl-uswst.accelrys.net:636
++ echo 'objectClass: top
'
++ sed /str/d
++ echo hbase/pp-hadoop-sec1.accelrys.net@ACCELRYS.NET
++ sed -e 's/\@ACCELRYS.NET//g'
++ echo -n '"AaBb=REDACTED-"'
++ iconv -f UTF8 -t UTF16LE
++ base64 -w 0
SASL/GSSAPI authentication started
SASL username: pp-hadoop@ACCELRYS.NET
SASL SSF: 0
ldap_add: No such attribute (16)
	additional info: 00000057: LdapErr: DSID-0C090B38, comment: Error in attribute conversion operation, data 0, vece

>>

[#2]

Found a fix, albeit a non-optimal one.

In the script gen_credentials_ad.sh I hard-coded the AD account properties

set -e
if [ "$SET_ENCRYPTION_TYPES" = "true" ]; then
ldapmodify -vH "$AD_SERVER" $SIMPLE_PWD_STR <<-%EOF
dn: $DIST_NAME
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
distinguishedName: $DIST_NAME
sAMAccountName: $USER
servicePrincipalName: $(echo $PRINC | sed -e "s/\@$CMF_REALM//g")
userPrincipalName: $PRINC
unicodePwd:: `echo -n "\"$PASSWD\"" | iconv -f UTF8 -t UTF16LE| base64 -w 0`
accountExpires: $ACCOUNTEXPIRES
userAccountControl: $USERACCOUNTCONTROL
%EOF

It would be nice for the script to be fixed by Cloudera.

[#3]

Hello Joe,

I didn't have a chance to reply to your original post, but that AD error was a bit unclear anyway. 

What did you end up identifying as the cause and what solution, exactly, did you implement?

It appears you removed the msds-supportedEncryptionTypes attribute.  

If so, what version of Windows/AD are you using?  We have added that code in Cloudera Manager 5.8.0 so that, if desired, Cloudera Manager will create the principal objects in Active Directory along with support for AES128 and AES256 encryption types.  You can opt out of this by making sure that Active Directory Set Encryption Types is set to the default of false in Administration --> Settings --> Kerberos

This is off by default, so it must have been checked at some point.  If false, the msds-supportedEncryptionTypes is not set.

Another question is why your Active Directory schema does not support that attribute.

Regards,

Ben

[#4]

As I recollect the msds-supportedEncryptionTypes was not the problem.  I did have the Active Directory Set Encryption Types checked however.  I went ahead and unchecked it.

I could be wrong, but the issue seems to be that in the original script the snippet

ldapmodify -H "$AD_SERVER" $SIMPLE_PWD_STR <<-%EOF
dn: $DIST_NAME
changetype: add
$(echo "$OBJECTCLASSES" | sed '/str/d')
distinguishedName: $DIST_NAME
....

the $OBJECTCLASSES comma delimited string isn't being handled correctly.  Once I hardcoded it, it worked fine.

I'm trying to get answer about the AD version.  I'm having to work through these issues with my IT department.

Is there a way to reinitalize things to regenerate the credentials again?  I'd be happy to help try to debug the original gen_credentials_ad.sh script.

[#5]

Very interesting.  I see that the output you have only has objectClass=top when the default is:

Active Directory Account Properties

accountExpires=0,objectClass=top,objectClass=person,objectClass=organizationalPerson,objectClass=user

If you have a little blue arrow near the "Active Directory Account Properties" configuration in the Kerberos Settings, click that to return to the default.

That said, you can regenerate credentials by shutting down your cluster and then checking all principals and clicking the Regenerate Selected button.  If you don't have Active Directory Delete Accounts on Credential Regeneration checked in your Kerberos Settings, you'll need to manually delete the principal objects from AD first.

I don't think this is an outright bug, but it would be nice to know what is going on with the objectclasses list.

[#6 · Accepted solution]

OK.  I switched back to the original script and put the default for the Account Properties.  I was able to regenerate no problem.  The encryption types is not checked.  Perhaps that was the problem then.  Haven't heard back yet about AD version(s).

[#7]

Hello

I'm having this same issue... I think

see I'm using active directory... and I'm using the encryption type:  aes256-cts-hmac-sha1-96

only... I'm running kinit on the server and everything looks to be good... then I try to generate the missing credentials which are all of then but I'm getting this:

Kerberos encryption types configuration is invalid or missing. 

as I said the encryption type is indeed set up, I also checked the file krbg5.conf and everythig looks good, in the AD I'm forcing the AES types within a policy.

my guess is that the encryption type is wrong... but I'm really lost .

any ideas?

thanks!!

[#8]

well... I resolved this by adding :
-aes256-cts-hmac-sha1-96
-aes128-cts
only those 2...
hope this help!