How do I set up Pig to use HCatalog?

rmckissick — Sat, 16 Jul 2016 06:50:22 GMT

I want Pig to use HCatalog in my Oozie workflow

Re: How do I set up Pig to use HCatalog?

aervits — Sat, 16 Jul 2016 07:06:37 GMT

Rita, please see my example https://github.com/dbist/oozie/tree/master/apps/hcatalog

Re: How do I set up Pig to use HCatalog?

bandalora — Sat, 16 Jul 2016 07:34:26 GMT

Here's another possibility, from the HDP 2.4.2 Release Notes:

Configuring Pig Scripts to Use HCatalog in Oozie Workflows

To access HCatalog with a Pig action in an Oozie workflow, you need to modify configuration information to point to the Hive metastore URIs.

There are two methods for providing this configuration information. Which method you use depends upon how often your Pig scripts access the HCatalog.

Configuring Individual Pig Actions to Access HCatalog

If only a few individual Pig actions access HCatalog, do the following:

Identify the URI (host and port) for the Thrift metastore server.

a. In Ambari, click Hive > Configs > Advanced.

b. Make note of the URI in the hive.metastore.uris field in the General section.

This information is also stored in the hive.default.xml file.

2. Add the following two properties to the <configuration> elements in each Pig action.

Note: Replace [host:port(default:9083)] in the example below with the host and port for the Thrift metastore server.

<configuration>
    <property>
	<name>hive.metastore.uris</name>
        <value>thrift://[host:port(default:9083)]</value>
	<description>A comma separated list of metastore uris the client can use to contact the metastore server.</description>
    </property>
    <property>
	<name>oozie.action.sharelib.for.pig</name>       
	<value>pig,hive,hcatalog</value>    
	<description>A comma separated list of libraries to be used by the Pig action.</description>
    </property>
</configuration>

Configuring All Pig Actions to Access HCatalog

If all of your Pig actions access HCatalog, do the following:

Add the following line to the job.properties files, located in your working directory:

oozie.action.sharelib.for.pig=pig,hive,hcatalog

2. Identify the URI (host and port) for the Thrift metastore server.

a. In Ambari, click Hive > Configs > Advanced.

b. Make note of the URI in the hive.metastore.uris field in the General section.

This information is also stored in the hive.default.xml file.

3. Add the following property to the <configuration> elements in each Pig action.

Note: Replace [host:port(default:9083)] in the example below with the host and port for the Thrift metastore server.

<configuration>
    <property>
	<name>hive.metastore.uris</name> 
	<value>thrift://[host:port(default:9083)]</value>
	<description>A comma separated list of metastore uris the client can use to contact the
metastore server.</description>
    </property>
</configuration>

question How do I set up Pig to use HCatalog? in Archives of Support Questions (Read Only)

How do I set up Pig to use HCatalog?

Re: How do I set up Pig to use HCatalog?

Re: How do I set up Pig to use HCatalog?