question Re: Phoenix Query Server Connection URL example? in Support Questions

Phoenix Query Server Connection URL example?

sunile_manjee — Tue, 26 Jul 2016 04:35:33 GMT

I am looking for an example PQS connection url which has kerberos params. My cluster is kerberized and the example on the apache phoenix site shows this as example:

jdbc:phoenix:thin:url=<scheme>://<server-hostname>:<port>[...]

The site documents the kerberos params. However I would like to see a working example. Any example appreciated.

Extra Info:

Configurations relating to server connecting to a secure cluster.
Property	Description	Default
hbase.security.authentication	When set to "kerberos", the server will attempt to log in before initiating Phoenix connections.	Specified hbase-default.xml
phoenix.queryserver.keytab.file	The key to look for keytab file.	unset
phoenix.queryserver.kerberos.principal	The kerberos principal to use when authenticating.	unset
phoenix.queryserver.dns.nameserver	The DNS hostname	default
phoenix.queryserver.dns.interface	The name of the network interface to query for DNS.	default

Re: Phoenix Query Server Connection URL example?

ssoldatov — Tue, 26 Jul 2016 06:32:06 GMT

Those parameters are for the QueryServer configuration. They need to be added to the hbase-site.xml, so Query Server will be able to connect to the HBase. As far as I know there is no kerberos configuration parameters in the jdbc connection string for PQS

Re: Phoenix Query Server Connection URL example?

sunile_manjee — Fri, 29 Jul 2016 12:14:58 GMT

@ssoldatov I don't understand what you mean when a kerberos princple is different per user. This needs to be defined as it is today for the jdbc connection as @Constantin Stanca provided. I am looking for example of thin jdbc url example with kerberos.

Re: Phoenix Query Server Connection URL example?

ssoldatov — Fri, 29 Jul 2016 12:39:02 GMT

@Sunile Manjee PQS works like most of others hadoop services. It requires its own keytab and principal to access hdfs/hbase. (please note that documentation says Configurations relating to server connecting to a secure cluster.) Once you configured PQS kerberos keytab/principal any client may work with PQS without restrictions. No client auth is supported at the moment. It will be added in HDP 2.5 release. More details can be found at https://issues.apache.org/jira/browse/PHOENIX-2792

Re: Phoenix Query Server Connection URL example?

sunile_manjee — Fri, 29 Jul 2016 21:49:02 GMT

@ssoldatov I am aware it is in 2.5 and that is exatly where I am testing this. I need example URL

Re: Phoenix Query Server Connection URL example?

ssoldatov — Sat, 30 Jul 2016 02:14:43 GMT

This implementation is using kerberos ticket cache, so you need to use kinit before running client. No changes in the URL is required. Also make sure that the hbase-site.xml which is in the classpath of the client has hbase.security.authentication set to kerberos

Re: Phoenix Query Server Connection URL example?

elserj — Mon, 01 Aug 2016 21:23:52 GMT

There are two sides here. The documentation that you listed and @ssoldatov confirmed are accurate for PQS to connect to HBase. The other side, which is likely missing as "official" Apache Phoenix documentation, is the thin-client configuration properties.

These properties are presently available at http://calcite.apache.org/avatica/docs/client_reference.html. The sqlline-thin.py script will automatically configure them for you, but you would have to provide them when using the thin JDBC driver directly. In practice, it would look something like the following when you have already performed a Kerberos login

jdbc:phoenix:thin:url=<scheme>://<server-hostname>:<port>;authentication=SPNEGO

Alternatively, you can provide a principal and keytab which the thin driver will use to login automatically:

jdbc:phoenix:thin:url=<scheme>://<server-hostname>:<port>;authentication=SPNEGO;principal=my_user;keytab=/home/my_user/my_user.keytab

Re: Phoenix Query Server Connection URL example?

sunile_manjee — Tue, 02 Aug 2016 03:58:17 GMT

@Josh Elser That is exactly what I needed. Now I need to test it. If I run into issues will open another HCC question. Thanks again Josh!

Re: Phoenix Query Server Connection URL example?

christopher_bri — Thu, 03 Nov 2016 16:46:59 GMT

Hi Josh,

I'm using HDP 2.5 (upgraded from 2.4.2) and have attempted to follow the above instructions but it's not working from Java client or on the master (./sqline.py works but not "thin").

I've run kinit,

kinit -k -t /etc/security/keytabs/hbase.headless.keytab hbase-cluster1

Then I run (obfuscated),

./sqlline-thin.py http://b3e073.*****.com:8765;authentication=SPNEGO

I also tried,

./sqlline-thin.py http://localhost:8765;authentication=SPNEGO

The (obfuscated) output is,

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.0.0-1245/phoenix/phoenix-4.7.0.2.5.0.0-1245-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.0.0-1245/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/11/03 05:37:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting property: [incremental, false]
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:thin:url=http://b3e073.*****.com:8765;serialization=PROTOBUF;authentication=SPNEGO none none org.apache.phoenix.queryserver.client.Driver
Connecting to jdbc:phoenix:thin:url=http://b3e073.*****.com:8765;serialization=PROTOBUF;authentication=SPNEGO
java.lang.RuntimeException: Failed to execute HTTP Request, got HTTP/404
at org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientSpnegoImpl.send(AvaticaCommonsHttpClientSpnegoImpl.java:148)
at org.apache.calcite.avatica.remote.RemoteProtobufService._apply(RemoteProtobufService.java:44)
at org.apache.calcite.avatica.remote.ProtobufService.apply(ProtobufService.java:81)
at org.apache.calcite.avatica.remote.Driver.connect(Driver.java:175)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:157)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:203)
at sqlline.Commands.connect(Commands.java:1064)
at sqlline.Commands.connect(Commands.java:996)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:804)
at sqlline.SqlLine.initArgs(SqlLine.java:588)
at sqlline.SqlLine.begin(SqlLine.java:656)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
at org.apache.phoenix.queryserver.client.SqllineWrapper$1.run(SqllineWrapper.java:78)
at org.apache.phoenix.queryserver.client.SqllineWrapper$1.run(SqllineWrapper.java:75)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.phoenix.queryserver.client.SqllineWrapper.main(SqllineWrapper.java:75)
sqlline version 1.1.8

any idea where I'm going wrong?

thanks, Chris.

Re: Phoenix Query Server Connection URL example?

christopher_bri — Thu, 03 Nov 2016 17:03:11 GMT

PHOENIX-2792 says fixed in 4.8.0, HDP 2.5 has 4.7.0.2.5.0.0-1245, are Hortonworks confident this feature made it into the HDP 2.5 distribution?

Re: Phoenix Query Server Connection URL example?

elserj — Thu, 03 Nov 2016 21:57:02 GMT

Hey Chris -- using localhost definitely won't work. You must use the FQDN in the URL you supply (which must match the FQDN in the principal which PQS is using. I'm not sure why you would be getting an HTTP/404 though. I'd take a look at the PQS logs to see if anything is there. Would be best to not piggy-back on this issue and ask a new question instead. Feel free to tag me there so I'm sure to see it.

Re: Phoenix Query Server Connection URL example?

christopher_bri — Fri, 04 Nov 2016 02:03:02 GMT

thanks will do.

BTW - last piggy-back! - I checked the log (/usr/hadoop/log/hbase/phoenix-hbase-server.log) its giving this,

2016-11-03 15:00:54,679 WARN org.apache.phoenix.shaded.org.eclipse.jetty.security.SpnegoLoginService:
GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)
        at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:856)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
        at sun.security.jgss.spnego.SpNegoContext.GSS_acceptSecContext(SpNegoContext.java:906)
        at sun.security.jgss.spnego.SpNegoContext.acceptSecContext(SpNegoContext.java:556)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)

Re: Phoenix Query Server Connection URL example?

elserj — Fri, 04 Nov 2016 02:34:21 GMT

Sadly, "Checksum failed" could be for numerous different reasons. You can try to set "-Dsun.security.spnego.debug=true" in PHOENIX_QUERYSERVER_OPTS in hbase-env.sh for lots of debug after a restart of PQS. This will require careful reading and observation (based on principals and hostnames) to figure out what went wrong though.

Re: Phoenix Query Server Connection URL example?

shiwang — Wed, 08 Feb 2017 08:39:52 GMT

I have a question regarding to this...

If I wanna connect to a https url, and I specify the truststore=** after the url, still it will show error

java.lang.RuntimeException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

What else am i missing?

Re: Phoenix Query Server Connection URL example?

elserj — Thu, 09 Feb 2017 00:11:37 GMT

There is no version of Avatica released which has the ability to specify the truststore.

Re: Phoenix Query Server Connection URL example?

kaolian — Fri, 24 Feb 2017 07:31:04 GMT

Which O/S you are using for the KDC server? Is it same as the other cluster servers? If you also use Ranger encryption, there something else step you need to do. try kinit -V -k -t /etc/security/keytabs/spnego.service.keytab HTTP/YOUR_SERVER_DOMAIN@EXAMPLE.COM and hbase shell. if you scan the table like 'scan 'SYSTEM.CATALOG'. then try thin client once again.

Re: Phoenix Query Server Connection URL example?

sjanghel — Wed, 07 Feb 2018 02:41:08 GMT

Anyone has a working solution for this I tried pheonixdb but nothing suggested above works currently or I am not using the configs properly

Re: Phoenix Query Server Connection URL example?

elserj — Wed, 07 Feb 2018 02:52:06 GMT

To the best of my knowledge, python-phoenixdb (https://bitbucket.org/lalinsky/python-phoenixdb) does not have support for SPENGO which is implicitly required when a cluster has Kerberos authentication enabled. Please open your own question if you have more information to share.

Re: Phoenix Query Server Connection URL example?

bavisetti — Fri, 14 Apr 2023 10:49:56 GMT

hi @elserj ,

can you share hbase connection string with krb5 configurations?

thanks,

jyothsna