Support Questions
Find answers, ask questions, and share your expertise

How to store PIG result into HIVE table by using OOZIE workflow ?

Contributor

I want to perform Oozie for store PIG result into HIVE table.

In that,

  • PIG job would load files from HDFS directory and store result into specific HDFS directory.
  • HIVE job would create an external table and load PIG result as input.

How could I do this ? Can anyone suggest me an example for Oozie workflow.

Thank you,

Jay.

4 REPLIES 4

Re: How to store PIG result into HIVE table by using OOZIE workflow ?

Super Collaborator

Hi @JAy PaTel,

You may use the Hcatolg's HCatStorer to load the data into Hive Table,

but firstly make sure that you initialize the pig session with "-useHCatalog" as an example "pig -useHCatalog"

there after, lets say your results are stored in a variable vData, to load the same into Hive

STORE vData INTO '<HiveTable>'
   USING org.apache.hive.hcatalog.pig.HCatStorer();

as one will load the data into Hive table, and this can be called from pigaction form Oozie.

to pass -useHCatalog you may use argument attribute in the workflow XML.

<argument>-useHCatalog</argument>

More on pig to access Hive data (reading from Hive table and writing into Hive table can be fount at - https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore

Oozie reference for pig action can be found at - https://oozie.apache.org/docs/3.2.0-incubating/WorkflowFunctionalSpec.html#a3.2.3_Pig_Action

Hope this Helps !!

Re: How to store PIG result into HIVE table by using OOZIE workflow ?

Contributor

Hi @bkosaraju, thank you for responding.

I tried to run Pig Script on Grunt shell before executing the workflow, but 'HCatStorer' is not working on grunt shell.
https://community.hortonworks.com/questions/190891/problem-with-import-pig-output-into-hive-table-us...

Regrads,
Jay.

Re: How to store PIG result into HIVE table by using OOZIE workflow ?

New Contributor

Did you solved this?

Could you please provide some inputs I have the same question

Re: How to store PIG result into HIVE table by using OOZIE workflow ?

Community Manager

@Hadoop107, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.


Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community: