Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Phoenix Storage in Pig

avatar
Contributor

What jar(s) do I need to register or which command line options do I need to pass to Pig so that Phoenix + Pig integration works?

I'm using HDP 2.3.2. I've created a table in Phoenix and I would like to load sample data into a Phoenix table from a Hive table via a Pig script. When I try to store data into Phoenix I get "Could not resolve org.apache.phoenix.pig.PhoenixHBaseStorage using imports ...".

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Michael Young register the Phoenix client jar from /usr/hdp/current/phoenix-client in pig script

View solution in original post

6 REPLIES 6

avatar
Master Mentor

@Michael Young you need access to the Phoenix client jar, more reference Here

avatar
Master Mentor

@Michael Young register the Phoenix client jar from /usr/hdp/current/phoenix-client in pig script

avatar
Contributor

Thank you that is what I was looking for. It wasn't clear to me which jar file I needed to register.

avatar
Master Mentor

@Michael Young try it out and let us know. Post ayour pig script here so that others with same question can reference it. Surprised myself that it was not included in the project docs.

avatar
Contributor

I create a Phoenix table using the phoenix-sqlline client via:

create table mytable (id varchar not null primary key, mycolumn varchar);

Here is the pig script I used to attempt to load the data from Hive/HCatalog to Phoenix.

REGISTER /usr/hdp/2.3.2.0-2950/phoenix/phoenix-4.4.0.2.3.2.0-2950-client.jar

A = LOAD 'default.mytable' USING org.apache.hive.hcatalog.pig.HCatLoader();

STORE A INTO 'hbase://mytable' using org.apache.phoenix.pig.PhoenixHBaseStorage('hbaseserver', '-batchSize 1000');

The pig script is run via:

pig -x tez -useHCatalog mypigscript.pig

I no longer get a "Could not resolve ..." error. However, I'm running into another error where the table in Phoenix doesn't exist.

avatar
Contributor

I had an issue with ZK.

I stopped HBase via Ambari. I ran "hbase clean --cleanZk". I then started HBase via Ambari. Now the Pig script is loading data.

@Neeraj Sabharwal @Josh ElserThanks for helping to resolve the issue via another post.