Cloudera 5.16.1 Kerberos Import Credentials Script Failing with Active Directory

[#1]

Hi,

I'm running Cloudera 5.16.1 on CentOS 7 and OpenJDK8

I enabled TLS/SSL on the Cloudera Manager (level 1 - level3) with Self-Signed certificate and then moved to enable Kerberos with AD.

I followed the docs but enabling Kerberos is failing when trying to import credentials.

/usr/share/cmf/bin/import_credentials.sh failed with exit code 1 and output of <<
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ KEYTAB_OUT=/var/run/cloudera-scm-server/cmf1018223695564634823.keytab
+ USER=cdhadmin@<XXXXX>.COM
+ PASSWD=REDACTED
+ KVNO=1
+ SLEEP=0
+ RHEL_FILE=/etc/redhat-release
+ '[' -f /etc/redhat-release ']'
+ set +e
+ grep Tikanga /etc/redhat-release
+ '[' 1 -eq 0 ']'
+ '[' 0 -eq 0 ']'
+ grep 'CentOS release 5' /etc/redhat-release
+ '[' 1 -eq 0 ']'
+ '[' 0 -eq 0 ']'
+ grep 'Scientific Linux release 5' /etc/redhat-release
+ '[' 1 -eq 0 ']'
+ set -e
+ '[' -z /var/run/cloudera-scm-server/krb51430682016564011407.conf ']'
+ echo 'Using custom config path '\''/var/run/cloudera-scm-server/krb51430682016564011407.conf'\'', contents below:'
+ cat /var/run/cloudera-scm-server/krb51430682016564011407.conf
+ IFS=' '
+ read -a ENC_ARR
+ for ENC in '"${ENC_ARR[@]}"'
+ echo 'addent -password -p cdhadmin@<XXXXX>.COM -k 1 -e rc4-hmac'
+ ktutil
+ '[' 0 -eq 1 ']'
+ echo REDACTED
+ echo 'wkt /var/run/cloudera-scm-server/cmf1018223695564634823.keytab'
+ chmod 600 /var/run/cloudera-scm-server/cmf1018223695564634823.keytab
+ kinit -k -t /var/run/cloudera-scm-server/cmf1018223695564634823.keytab cdhadmin@<XXXXX>.COM
+ '[' true '!=' true ']'
++ mktemp /tmp/cm_ldap.XXXXXXXX
+ LDAP_CONF=/tmp/cm_ldap.lZPuleq0
+ echo 'TLS_REQCERT     never'
+ echo 'sasl_secprops   minssf=0,maxssf=0'
+ export LDAPCONF=/tmp/cm_ldap.lZPuleq0
+ LDAPCONF=/tmp/cm_ldap.lZPuleq0
+ set +e
+ ldapsearch -LLL -H ldaps://<xxxxx>.<xxxxx>.com:636 -b OU=cdh-kerberos,OU=CDH,DC=<xxxxx>,DC=com userPrincipalName=cdhadmin@<XXXXX>.COM
ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
+ '[' 255 -ne 0 ']'
+ echo 'ldapsearch did not work with SASL authentication. Trying with simple authentication'
+ ldapsearch -LLL -H ldaps://<xxxxx>.<xxxxx>.com:636 -b OU=cdh-kerberos,OU=CDH,DC=<xxxxx>,DC=com -x -D cdhadmin@<XXXXX>.COM -w REDACTED userPrincipalName=cdhadmin@<XXXXX>.COM
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
+ '[' 255 -ne 0 ']'
+ echo 'Failed to do ldapsearch.'
+ echo 'Please make sure Active Directory configuration is correctly specified and LDAP over SSL is enabled.'
+ exit 1

I verified that LDAPS is enabled for Active Directory and verified all settings again but could not solve the issue.

I ran the below command manually on the cloudera server and it worked fine (got output):

ldapsearch -LLL -H ldaps://<xxxxx>.<xxxxx>.com:636 -b OU=cdh-kerberos,OU=CDH,DC=<xxxxx>,DC=com -x -D cdhadmin@<XXXXX>.COM -w <PASSWORD>

One thing I should mention is that I did not edit the krb5.conf file and checked the box Manage krb5.conf through Cloudera Manager in the settings.

Please assist / advise.

[#2]

@khjomaa The error message is complaining about the LDAP(S) issue, though as you said you already tried the below command and this worked.

ldapsearch -LLL -H ldaps://<xxxxx>.<xxxxx>.com:636 -b OU=cdh-kerberos,OU=CDH,DC=<xxxxx>,DC=com -x -D cdhadmin@<XXXXX>.COM -w <PASSWORD>

 I would like to ask the output of below command form CM server host which will give you a clear picture of LDAP over SSL is enabled from AD side or not. If openssl shows no peer certificate is returned. This indicates that the port is NOT listening via TLS:

# openssl s_client -showcerts -connect <xxxxx>.<xxxxx>.com:636

 Also for the Manage krb5.conf through Cloudera Manager it will be good to enable it if you want to manage krb5.conf file without any intervention and smoothly. 

Cheers,

[#3 · Accepted solution]

Hi,

Thanks for your reply.

After investigating the issue I found that command (ldapsearch) failed because user cloudera-scm does not have the proper permissions.

So as a work around I added sudo before ldap commands in the following scripts:

/usr/share/cmf/bin/import_credentials.sh

/usr/share/cmf/bin/gen_credentials_ad.sh

and then everything worked fine. 

[#4]

Glad to hear that issue is resolved. Please close this thread by marking this as solved.

Cheers,
Thanks for contributing in Cloudera Community.