Support Questions

Find answers, ask questions, and share your expertise

Phoenix - Query Timeout

avatar
Contributor

I have a table in HBase created via Phoenix. The table has approxmiately 20 million records. I'm connecting to Phoenix via:

phoenix-sqlline.py hbasemaster:2181:/hbase-unsecure

I'm trying to run a count as follows:

select count(columnname) from tablename;

When I run that SQL, Phoenix reports a timeout

org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:

java.net.SocketTimeoutException: callTimeout=60000, callDuration=60317: row ....

I've tried changing the hbase.rpc.timeout via Ambari, but that doesn't seem to be the issue. The default timeout in Ambari was set to 1m30s and I changed it to 2m. The timeout reported by Phoenix is 60s before and after the change, so I don't think that's the culprit anyway.

What setting do I need to change to allow for longer running queries? Is there something else that I should be looking at?

1 ACCEPTED SOLUTION

avatar
Super Collaborator

There is an odd environment variable used for the HBase configuration directory. Try setting HBASE_CONF_PATH=/etc/hbase/conf in your shell session and see if the timeout value gets picked up.

View solution in original post

16 REPLIES 16

avatar
Super Collaborator

There is an odd environment variable used for the HBase configuration directory. Try setting HBASE_CONF_PATH=/etc/hbase/conf in your shell session and see if the timeout value gets picked up.

avatar
Contributor

This was the issue. Well more specifically HBASE_CONF_DIR. Running phoenix_utils.py showed HBASE_CONF_PATH and HBASE_CONF_DIR as equal to ".". I set my HBASE_CONF_DIR environment variable to to use "/etc/hbase/conf" and now the query seems to work ok.

avatar
Super Guru

FWIW, this should be fixed in recent versions of Apache Phoenix and HDP.

avatar
Contributor

Hi @Terry Padgett @Josh Elser,

I am using DBVisulizer to connect to Phoenix. Mine is a kerberos enabled cluster and getting below error messages. Can suggestions?

An error occurred while establishing the connection: Long Message: callTimeout=600000, callDuration=620131: Details: Type: org.apache.phoenix.exception.PhoenixIOException Error Code: 101 SQL State: 08000

Not sure how I can pass conf directory for DBVisulizer (added all conf files to Resoucres directory of DbViz but didn't helped)

avatar
New Contributor

Hi All,

I set phoenix timeout is 3hr, rpc 3hr and query alive is also 3hr but for bigger query whenever phoenix calltimeout occurs my hbase region server also goes down, I don not know what is happening.

Please help me out on this. @Terry Padgett

avatar
Contributor

avatar
New Contributor

Frustrating that that link is hidden behind a 'paywall'.  I have an account but I am not allowed to view without contacting sales