question Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster in Support Questions

HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

qiwang — Mon, 19 Aug 2019 08:57:44 GMT

HDP 2.5 secured cluster with Zeppelin and HBase installed. I could connect to Phoenix using the sqlline.py utility using the following parameter.

/usr/hdp/current/phoenix-client/bin/sqlline.py <zk1>,<zk2>,<zk3>:2181:/hbase-secure:hbase@DOMAIN.COM:/etc/security/keytabs/hbase.headless.keytab

However when I try to use similar parameter for jdbc conn string in phoenix, I got

null
org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=1, exceptions:
Wed Jan 18 15:13:11 UTC 2017, RpcRetryingCaller{globalStartTime=1484752390908, pause=100, retries=1}, org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.io.IOException: Broken pipe

class org.apache.zeppelin.interpreter.InterpreterException

The jdbc interpreter settings for Phoenix are

Do I need to create phoenixuser principal and use that keytab instead? and what is phoenix.user and phoenix.password?

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

elserj — Thu, 19 Jan 2017 01:13:27 GMT

org.apache.hadoop.hbase.MasterNotRunningException

It would appear that HBase is not actually running. Can you verify that HBase is healthy? Can you interact with HBase via `hbase shell` (e.g. create a table, add data to it, delete the table)?

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

qiwang — Thu, 19 Jan 2017 04:28:29 GMT

@Josh Elser

HBase is running fine. I cold pull the table list and look into "atlas_titan" tables content

[root@qwang-hdp5 logs]# hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.1.2.2.5.3.0-37, rcb8c969d1089f1a34e9df11b6eeb96e69bcf878d, Tue Nov 29 18:48:22 UTC 2016

hbase(main):001:0> list
TABLE
ATLAS_ENTITY_AUDIT_EVENTS
SYSTEM.CATALOG
SYSTEM.FUNCTION
SYSTEM.SEQUENCE
SYSTEM.STATS
atlas_titan
driver_dangerous_event
my_table
8 row(s) in 0.5710 seconds

hbase(main):003:0> scan 'atlas_titan'

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

elserj — Thu, 19 Jan 2017 04:55:00 GMT

Can you please try to create a table? This specifically uses the Master whereas scanning a table can be done by talking to a RegionServer.

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

ssoldatov — Thu, 19 Jan 2017 05:05:06 GMT

I bet that HBase is healthy. Was able to reproduce it locally. The same exception running test from zeppelin with perfectly working sqlline (as well as hbase shell).

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

qiwang — Thu, 19 Jan 2017 06:58:24 GMT

@Josh Elser

HBase is up and running fine. I can create table in both hbase shell and sqlline.py

hbase(main):001:0> create 'my_table1', {NAME =>'cf1'}, {NAME =>'cf2'}
0 row(s) in 2.9560 seconds


=> Hbase::Table - my_table1
hbase(main):002:0> put 'my_table1', 'rowkey01', 'cf1:c1', 'test value'
0 row(s) in 0.6850 seconds


hbase(main):003:0> get 'my_table1', 'rowkey01'
COLUMN                          CELL
 cf1:c1                         timestamp=1484780190152, value=test value
1 row(s) in 0.0780 seconds

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

qiwang — Thu, 19 Jan 2017 06:59:28 GMT

@Sergey Soldatov

Are you running into the same problem on a secured cluster or non-secured one?

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

ssoldatov — Thu, 19 Jan 2017 07:22:12 GMT

@Qi Wang

Well, to fix it you will need to proceed few steps:

1. In the jdbc interpreters configuration you need to remove default artifacts for phoenix and hbase (phoenix-4.7... and hbase-1.1.3...).

2. add new artifact for phoenix-client.jar (just provide the path /usr/hdp/current/phoenix-client/phoenix-client.jar)

3. You need to set up interpreter to properly find hbase configuration dir. It could be done by adding to the end of zeppelin-env.sh

export CLASSPATH=$CLASSPATH:/etc/hbase/conf

using Ambari and after restart of Zeppelin service everything is supposed to work

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

elserj — Thu, 19 Jan 2017 08:59:08 GMT

Sorry for the tangent, Qi. Glad Sergey was here to steer you in a better direction 🙂

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

qiwang — Thu, 19 Jan 2017 11:06:42 GMT

@Sergey Soldatov

It works after following your steps. It would be nice to figure out how to do step 3 in ambari if possible, always worry some later ambari update would wipe the change out. thanks a lot for your help!

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

ssoldatov — Thu, 19 Jan 2017 13:20:39 GMT

I checked the script more carefully. It looks like adding

export CLASSPATH=$CLASSPATH:/etc/hbase/conf

to zeppelin-env.sh using Zeppelin configuration in Ambari would be enough.

Re: HDP 2.5 How to connect Zeppelin to Phoenix in secured cluster

qiwang — Fri, 20 Jan 2017 11:33:48 GMT

@Sergey Soldatov

Add it in ambari at the end of "Advanced zeppelin-env" -> "zeppelin_env_content" worked perfectly.