Reply
New Contributor
Posts: 5
Registered: ‎11-29-2017

cdh5.14.2 + Phoenix QueryServer 4.14.1 + Kerberos

[ Edited ]

I'm trying to setup Apache Phoenix QueryServer in secure HBase environment.

My hbase-site.xml is:

<configuration>
  <property>
    <name>hbase.regionserver.wal.codec</name>
    <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>zk1,zk2,zk3</value>
  </property>
<property>
    <name>hbase.security.authentication</name>
    <value>kerberos</value>
</property>
<property>
    <name>phoenix.queryserver.keytab.file</name>
    <value>/usr/lib/apache-phoenix-queryserver/bin/phoenix.keytab</value>
</property>
<property>
    <name>phoenix.queryserver.kerberos.principal</name>
    <value>rwqueryserver/_HOST@FOO.BAR</value>
</property>
<property>
    <name>phoenix.queryserver.http.keytab.file</name>
    <value>/usr/lib/apache-phoenix-queryserver/bin/phoenix.keytab</value>
</property>
<property>
    <name>phoenix.queryserver.http.kerberos.principal</name>
    <value>rwqueryserver/_HOST@FOO.BAR</value>
</property>
</configuration>

Query server is starting without any problems.

2018-12-12 09:13:07,353 INFO org.apache.phoenix.queryserver.server.QueryServer: Login successful.

I checked KDC side and I can see logins from the principal used for server/client connections. No errors on KDC side as well. Thin client command is:

./sqlline-thin.py 'http://dns-of-query-server:8765;principal="rwqueryserver/dns-of-query-server@DATASYS.CF.WTF";keytab="/usr/lib/apache-phoenix-queryserver/bin/phoenix.keytab"'

I tried to use user's principal and server's principal - situation is the same.

java.lang.RuntimeException: Failed to execute HTTP Request, got HTTP/404

From the queryserver log:

2018-12-12 09:15:30,987 WARN org.apache.phoenix.shaded.org.eclipse.jetty.security.SpnegoLoginService: 
GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96)
        at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:856)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
        at sun.security.jgss.spnego.SpNegoContext.GSS_acceptSecContext(SpNegoContext.java:906)
        at sun.security.jgss.spnego.SpNegoContext.acceptSecContext(SpNegoContext.java:556)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
        at org.apache.phoenix.shaded.org.eclipse.jetty.security.SpnegoLoginService.login(SpnegoLoginService.java:137)
        at org.apache.phoenix.shaded.org.eclipse.jetty.security.authentication.LoginAuthenticator.login(LoginAuthenticator.java:61)
        at org.apache.phoenix.shaded.org.eclipse.jetty.security.authentication.SpnegoAuthenticator.validateRequest(SpnegoAuthenticator.java:99)
        at org.apache.phoenix.shaded.org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
        at org.apache.phoenix.shaded.org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
        at org.apache.phoenix.shaded.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.apache.phoenix.shaded.org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.apache.phoenix.shaded.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
        at org.apache.phoenix.shaded.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.apache.phoenix.shaded.org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
        at org.apache.phoenix.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.apache.phoenix.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
Caused by: KrbException: Invalid argument (400) - Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96
        at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:278)
        at sun.security.krb5.KrbApReq.<init>(KrbApReq.java:149)
        at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:108)
        at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:829)
        ... 19 more

When I'm trying to use non-existing principal I'm getting different errors. I checked JCE - it's installed.

jrunscript -e 'print (javax.crypto.Cipher.getMaxAllowedKeyLength("AES") >= 256);'
true

Can you advice anything ?

Master
Posts: 381
Registered: ‎07-01-2015

Re: cdh5.14.2 + Phoenix QueryServer 4.14.1 + Kerberos

Can you post your /etc/krb5.conf? And your setting of KDC what kind of ciphers are supported?
Highlighted
New Contributor
Posts: 5
Registered: ‎11-29-2017

Re: cdh5.14.2 + Phoenix QueryServer 4.14.1 + Kerberos

[libdefaults]
dns_lookup_kdc = false
dns_uri_lookup = false
ticket_lifetime = 24h
renew_lifetime = 7d
default_tgs_enctypes = "aes256-cts-hmac-sha1-96 -des -des3 -rc4 -camellia"
default_tkt_enctypes = "aes256-cts-hmac-sha1-96 -des -des3 -rc4 -camellia"

default_ccache_name = KEYRING:persistent:%{uid}
default_realm = FOO.BAR

[logging]
default = FILE:/var/log/krb5libs.log

[realms]
FOO.BAR = {
    kdc = krb-kdc001-server
    admin_server = krb-kdc001-server
}

From krb serever

[realms]
FOO.BAR = {
    max_life = 12h 0m 0s
    max_renewable_life = 7d 0h 0m 0s
    master_key_type = aes256-cts
    supported_enctypes = aes256-cts:normal
    dict_file = /usr/share/dict/words
}
Announcements

Currently incubating in Cloudera Labs:

Envelope
HTrace
Ibis
Impyla
Livy
Oryx
Phoenix
Spark Runner for Beam SDK
Time Series for Spark
YCSB