Thanks for the reply!  To be honest, I'm not 100% sure what I'm doing haha.  HDFS is failing to kinit when the service tries to come back up after enabling Kerberos auth.  The above is my attempt to reproduce the error on the command line, since I am not in control when the hdfs service tries to come up.  I forget if I attached this above - this is the service error from the Cloudera Manager when the HDFS service tries to come up: stdout: Fri Feb 23 14:52:47 UTC 2018 JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera using /usr/java/jdk1.7.0_67-cloudera as JAVA_HOME using 5 as CDH_VERSION using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE as CONF_DIR using as SECURE_USER using as SECURE_GROUP unlimited /usr/bin/kinit using hdfs/quickstart.cloudera@REALM.COM as Kerberos principal using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 as Kerberos ticket cache kinit was not successful. Fri Feb 23 14:52:49 UTC 2018 JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera using /usr/java/jdk1.7.0_67-cloudera as JAVA_HOME using 5 as CDH_VERSION using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE as CONF_DIR using as SECURE_USER using as SECURE_GROUP unlimited /usr/bin/kinit using hdfs/quickstart.cloudera@REALM.COM as Kerberos principal using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 as Kerberos ticket cache kinit was not successful. Fri Feb 23 14:52:51 UTC 2018 JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera using /usr/java/jdk1.7.0_67-cloudera as JAVA_HOME using 5 as CDH_VERSION using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE as CONF_DIR using as SECURE_USER using as SECURE_GROUP unlimited /usr/bin/kinit using hdfs/quickstart.cloudera@REALM.COM as Kerberos principal using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 as Kerberos ticket cache kinit was not successful. Fri Feb 23 14:52:54 UTC 2018 JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera using /usr/java/jdk1.7.0_67-cloudera as JAVA_HOME using 5 as CDH_VERSION using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE as CONF_DIR using as SECURE_USER using as SECURE_GROUP unlimited /usr/bin/kinit using hdfs/quickstart.cloudera@REALM.COM as Kerberos principal using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 as Kerberos ticket cache kinit was not successful. stderr: + export HADOOP_CLASSPATH=/usr/share/cmf/lib/plugins/event-publish-5.7.0-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-5.7.0.jar:/usr/share/cmf/lib/plugins/navigator/cdh57/audit-plugin-cdh57-2.6.0-shaded.jar + HADOOP_CLASSPATH=/usr/share/cmf/lib/plugins/event-publish-5.7.0-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-5.7.0.jar:/usr/share/cmf/lib/plugins/navigator/cdh57/audit-plugin-cdh57-2.6.0-shaded.jar + set -x + replace_conf_dir + find /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE -type f '!' -path '/var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/logs/' '!' -name '.log' '!' -name '*.keytab' '!' -name '*jceks' - exec perl -pi -e 's#CMF_CONF_DIR#/var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE#g' '{}' ';' + make_scripts_executable + find /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE -regex '.*\.\(py\|sh\)$' -exec chmod u+x '{}' ';' + '[' DATANODE_MAX_LOCKED_MEMORY '!=' '' ']' + ulimit -l + export HADOOP_IDENT_STRING=hdfs + HADOOP_IDENT_STRING=hdfs + '[' -n '' ']' + acquire_kerberos_tgt hdfs.keytab + '[' -z hdfs.keytab ']' + '[' -n hdfs/quickstart.cloudera@REALM.COM ']' + '[' -d /usr/kerberos/bin ']' + which kinit + '[' 0 -ne 0 ']' ++ id -u + export KRB5CCNAME=/var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 + KRB5CCNAME=/var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 + echo 'using hdfs/quickstart.cloudera@REALM.COM as Kerberos principal' + echo 'using /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 as Kerberos ticket cache' + kinit -c /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/krb5cc_497 -kt /var/run/cloudera-scm-agent/process/29-hdfs-NAMENODE/hdfs.keytab hdfs/quickstart.cloudera@REALM.COM kinit: Generic preauthentication failure while getting initial credentials + '[' 1 -ne 0 ']' + echo 'kinit was not successful.' + exit 1 For the record, I get the same error that I got above if I try to manually use the keytab in the `29-hdfs-NAMENODE` folder. Further, the reason that I added the `host` principal was because authenticating manually without it said it failed on needing the host principal.  So clearly my "reproduction" is not accurate, but I arrived at the same error (and an error that I do not encounter on other hosts), so I was hopeful to be onto something.  No luck so far though. The "Bad encryption type" has completely stymied me.  I have tried a bunch of the other available encryption types to no avail.  (Oh and myfile.keytab was created by me with the commands above since it was my attempt at a repro.  But as I mentioned, using the Cloudera-generated keytab also results in the same issue). ... View more

More logging enabled by setting the environment variable `KRB5_TRACE=/path/to/my/log` I changed the encryption type to aes128-cts here, and have tried a handful of others: [12979] 1519323668.876454: Getting initial credentials for host/quickstart.cloudera@REALM.COM [12979] 1519323668.876719: Looked up etypes in keytab: aes256-cts, aes128-cts, des3-cbc-sha1, rc4-hmac, des-hmac-sha1, des, des-cbc-crc [12979] 1519323668.876744: Sending request (222 bytes) to REALM.COM [12979] 1519323668.876787: Resolving hostname k-server [12979] 1519323668.877148: Initiating TCP connection to stream 192.168.123.2:88 [12979] 1519323668.877263: Sending TCP request to stream 192.168.123.2:88 [12979] 1519323668.877579: Received answer from stream 192.168.123.2:88 [12979] 1519323668.877629: Response was not from master KDC [12979] 1519323668.877647: Received error from KDC: -1765328359/Additional pre-authentication required [12979] 1519323668.877684: Processing preauth types: 136, 19, 2, 133 [12979] 1519323668.877692: Selected etype info: etype aes128-cts, salt "REALM.COMhostquickstart.cloudera", params "" [12979] 1519323668.877695: Received cookie: MIT [12979] 1519323668.877743: Retrieving host/quickstart.cloudera@REALM.COM from FILE:myfile.keytab (vno 0, enctype aes128-cts) with result: -1765328196/Bad encryption type [12979] 1519323668.877749: Preauth module encrypted_timestamp (2) (flags=1) returned: -1765328196/Bad encryption type [12979] 1519323668.877752: Produced preauth for next request: 133 [12979] 1519323668.877771: Getting initial credentials for host/quickstart.cloudera@REALM.COM [12979] 1519323668.877841: Looked up etypes in keytab: aes256-cts, aes128-cts, des3-cbc-sha1, rc4-hmac, des-hmac-sha1, des, des-cbc-crc [12979] 1519323668.877857: Sending request (222 bytes) to REALM.COM (master) And the keytab itself: Keytab name: FILE:myfile.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (aes256-cts-hmac-sha1-96) 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (aes128-cts-hmac-sha1-96) 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (des3-cbc-sha1) 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (arcfour-hmac) 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (etype 26) 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (etype 25) 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (des-hmac-sha1) 1 02/22/18 18:20:16 hdfs/quickstart.cloudera@REALM.COM (des-cbc-md5) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (aes256-cts-hmac-sha1-96) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (aes128-cts-hmac-sha1-96) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (des3-cbc-sha1) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (arcfour-hmac) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (etype 26) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (etype 25) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (des-hmac-sha1) 1 02/22/18 18:20:24 HTTP/quickstart.cloudera@REALM.COM (des-cbc-md5) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (aes256-cts-hmac-sha1-96) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (aes128-cts-hmac-sha1-96) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (des3-cbc-sha1) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (arcfour-hmac) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (etype 26) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (etype 25) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (des-hmac-sha1) 1 02/22/18 18:20:29 host/quickstart.cloudera@REALM.COM (des-cbc-md5) ... View more

I encountered the same issue running the docker-based cloudera quickstart on a AWS EC2 instance and accessing from my local computer.  The resolution was as follows:   Ensure that the hostname `quickstart.cloudera` resolves to the appropriate IP address.  I forced this by manually editing the `/etc/hosts` file locally (on a mac) Figure out / ensure that MySQL is listening on port 3306 within the container (`netstat -lntp` within the container to check this) Ensure that traffic at port 3306 on the host passes through to the container appropriately.  This amounts to adding `-p 3306:3306` to the `docker run` command   It seems that the Hue UI wants to access the MySQL database directly, which is a bit strange to me... in any case, that means that the client computer / browser needs to be able to access the MySQL database at `quickstart.cloudera:3306`.  Hope that helps, even if long after the fact! ... View more