Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Who agreed with this topic

Oozie - Spark Action - Hive

avatar
Contributor

I'm having problems getting a real hive context in a spark-scala application (jar) that is running as an Oozie spark action.   The spark app does write to hdfs folders just fine.  But it is unable to see the same tables that I see in the Hue Hive editor.  It seems to be pointing to creating a new metastore somewhere.  I have tried to include the hive-site.xml in various places but to no affect.  I've tried including it in the following locations:

 

  • The job xml for the spark action
  • The job xml for the workflow
  • A file tag in the workflow.xml for the spark action
  • etc.

 

I have run the code successfully many times in spark-shell.   I probably put it incorrectly in one of the locations. 

 

Any thoughts on what I am missing?

Who agreed with this topic