[CDH 5.8 Kerberos] Generate Missing Credentials Error

[#1]

Hi,

I am trying to enable kerberos in CDH 5.8 with os of ubuntu 14.04 

All are fine until I try to do the part of Generate Missing Credentials

Please find the error.

/usr/share/cmf/bin/gen_credentials.sh failed with exit code 127 and output of <<
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin:/usr/sbin
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin:/usr/sbin
+ export KRB5_CONFIG=/var/krb5kdc/krb5.conf
+ KRB5_CONFIG=/var/krb5kdc/krb5.conf
+ export KRB5_KDC_PROFILE=/var/krb5kdc/kdc.conf
+ KRB5_KDC_PROFILE=/var/krb5kdc/kdc.conf
+ export DEFKTNAME=FILE:/etc/krb5.keytab
+ DEFKTNAME=FILE:/etc/krb5.keytab
+ CMF_REALM=ACME.COM
+ KEYTAB_OUT=/tmp/cmf6659933762842902135.keytab
+ PRINC=kafka/sb-node1.example.com@ACME.COM
+ MAX_RENEW_LIFE=432000
+ KADMIN='kadmin -k -t /tmp/cmf622833402687775807.keytab -p cloudera-scm/admin@ACME.COM -r ACME.COM'
+ RENEW_ARG=
+ '[' 432000 -gt 0 ']'
+ RENEW_ARG='-maxrenewlife "432000 sec"'
+ '[' -z /var/krb5kdc/krb5.conf ']'
+ echo 'Using custom config path '\''/var/krb5kdc/krb5.conf'\'', contents below:'
+ cat /var/krb5kdc/krb5.conf
+ kadmin -k -t /tmp/cmf622833402687775807.keytab -p cloudera-scm/admin@ACME.COM -r ACME.COM -q 'addprinc -maxrenewlife "432000 sec" -randkey kafka/sb-node1.example.com@ACME.COM'
kadmin: relocation error: /usr/lib/x86_64-linux-gnu/libkadm5clnt_mit.so.9: symbol k5_buf_len, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference

[#2]

Yes, It means that I will use some environment variables which are set in bashrc.

Also, by running ldd on kadmin from script and outside, I found that there is differences between their results.

Command line - root

linux-vdso.so.1 => (0x00007ffdb1bad000)
libkadm5clnt_mit.so.10 => /usr/local/lib/libkadm5clnt_mit.so.10 (0x00007fd13679f000)
libkrb5.so.3 => /usr/local/lib/libkrb5.so.3 (0x00007fd1364ce000)
libk5crypto.so.3 => /usr/local/lib/libk5crypto.so.3 (0x00007fd13629d000)
libcom_err.so.3 => /usr/local/lib/libcom_err.so.3 (0x00007fd136099000)
libkrb5support.so.0 => /usr/local/lib/libkrb5support.so.0 (0x00007fd135e8d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd135ab2000)
libgssrpc.so.4 => /usr/local/lib/libgssrpc.so.4 (0x00007fd135895000)
libgssapi_krb5.so.2 => /usr/local/lib/libgssapi_krb5.so.2 (0x00007fd13564c000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fd135430000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd13522c000)
/lib64/ld-linux-x86-64.so.2 (0x0000564c783cb000)

CM

linux-vdso.so.1 => (0x00007fff078ce000)
libss.so.2 => /lib/x86_64-linux-gnu/libss.so.2 (0x00007fa458c31000)
libkadm5clnt_mit.so.9 => /usr/lib/x86_64-linux-gnu/libkadm5clnt_mit.so.9 (0x00007fa458a1e000)
libkrb5.so.3 => /usr/local/lib/libkrb5.so.3 (0x00007fa45874c000)
libk5crypto.so.3 => /usr/local/lib/libk5crypto.so.3 (0x00007fa45851c000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fa458318000)
libkrb5support.so.0 => /usr/local/lib/libkrb5support.so.0 (0x00007fa45810b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa457d46000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa457b42000)
libgssrpc.so.4 => /usr/local/lib/libgssrpc.so.4 (0x00007fa457924000)
libgssapi_krb5.so.2 => /usr/local/lib/libgssapi_krb5.so.2 (0x00007fa4576db000)
libcom_err.so.3 => /usr/local/lib/libcom_err.so.3 (0x00007fa4574d7000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fa4572bb000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa45709d000)
/lib64/ld-linux-x86-64.so.2 (0x000055aa9bce6000)'

[#3 · Accepted solution]

Thanks, could you also illustrate what you provide as env-vars, and why
they are required? The standard Kerberos setup documentation of Ubuntu does
not require any preset env-vars.

The ldd difference is what is driving the problem, and from the looks of it
there may be multiple kerberos libraries installed on the system, but am
uncertain how it has ended up that way. If you resolve the library trouble,
CM would be able to run the command normally.

[#4]

Thanks, you are right. I just discovered that there are two kadmin packages installed for unknown reason. Maybe it is because I changed the PATH variable once and installed the kadmin in other path where is different from the default path setting in CM.

I solved the problem with correcting the PATH variable and reinstalling the package. Once again, thank you for you help