Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Not able to read data from hive over phoenix table in spark-sql

I have created a table in hbase, over that a phoenix table, over that I have a hive table which I am trying to query in spark-sql prompt. No data, no exception it just stops the execution with no output data.

Any pointers are welcome and appreciated.

6 REPLIES 6

@sachin gupta,

You can try running the same query from hive shell and phoenix client to find out if this is the issue with spark or the query itself.

Let's assume that the table name is 'xyz';

1) Check if you are able to query data from hbase shell

> hbase shell
scan 'xyz'

2) Check if you are able to query data from phoenix client

> cd /usr/hdp/current/phoenix-client/bin/
> ./sqlline.py <zk-host>:2181/hbase-{un}secure ---> secure for kerberized, unsecure for non kerberized
select * from xyz;

3) Check if you are able to query data from hive shell

> hive
select * from xyz;

If you are getting valid data from all the 3 ways, then you should check the spark application logs to examine the reason for failure.

Thanks,

Aditya

@Aditya Sirna

thanks for your quick response. You got it right, I am getting all the output as expected from phoenix and hive shell. I have a question which logs should I check, thrift logs or of some other service. I have started spark-sql shell in debug mode and it seems spark is going to the base location of the hive table rather going to phoenix and looking for the data.

@sachin gupta,

You can check the ResourceManager for the application which ran for the query and check the logs for the application.

yarn logs -applicationId {app-id}

Thanks,

Aditya

sorry to tell you but it also not submitting any application. Anyways I want to use the power of spark and not run it as MR.

Super Collaborator

Phoenix has its own integration with Spark and using phoenix-spark module is preferable in this case and it would be much faster.

@Sergey Soldatov

Could you please give me some pointers or links which will help ?