Is it possible to integrate HDFS encryption with 3rd party KMS solutions? for example: Microsoft KMS I do see that there is Ranger KMS https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_Ranger_KMS_Admin_Guide/content/ch_ranger_kms_overview.html but we are looking into the option to integrate HDFS encryption with the existing KMS. ... View more

I am able to access gs without having kerberos ticket. I am guessing that it's normal but it would be nice to have a way to enforce kerberos auth for gs while accessing the GS from Hadoop. bash-4.1$ id uid=1023418093(hive) gid=1614812195(hadoop) ----------------------------------------------------------- bash-4.1$ kdestroy kdestroy: No credentials cache found while destroying cache ----------------------------------------------------------- bash-4.1$ klist klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_1023418093) ----------------------------------------------------------- bash-4.1$ hadoop fs -ls gs://dev/ 16/04/20 14:31:48 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.4.5-hadoop2 Found 1 items drwxrwxr-x - hive hive 0 2016-04-11 00:26 gs://dev/apps ----------------------------------------------------------- bash-4.1$ hadoop fs -ls / 16/04/20 14:30:56 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:413) at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:558) at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:373) ... View more