Reply
New Contributor
Posts: 1
Registered: ‎02-24-2014

Creating HIVE table through PIG Script

I have a use case where Pig UDF processes some data and generates either string or int etc. I need to store that back into a hive table.

 

For this, I have to create Hive table in advance for that schema and I am using HCatStorer() to store the same in hive. How can I change my pig script to let that create the hive table for me on the fly.

 

Please note that I would not easily know the schema in pig script, so cannot use HiveColumnarLoader as well.

Example: Pig script generates output as id (int), terms (a,b,c – list of string) and some keyvalue pair. It needs to go in hive as int, array<string>, map<string,string>

 

This may change in future, there can be more columns or different data types. How can I make this dynamic within the pig script?

Posts: 1,565
Kudos: 287
Solutions: 239
Registered: ‎07-31-2013

Re: Creating HIVE table through PIG Script

The HCatStorer's current assumption is that the specified table and database must pre-exist to its invocation.

It can be made to support self-creating said tables, but this would be a new feature request. Consider filing one at the upstream bug tracker: http://issues.apache.org/jira/browse/PIG.
Backline Customer Operations Engineer
Highlighted
New Contributor
Posts: 1
Registered: ‎10-30-2017

Re: Creating HIVE table through PIG Script

Is this feature available now?

Announcements