Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Load 2 fields only out 4 fields - Pig to Hive table

Load 2 fields only out 4 fields - Pig to Hive table

Explorer

Dear Experts,

 

Pls help advise.

 

Scenario: Store data from HDFS to Hive using Pig.

 

I have file that scooped from mySQL, which has 4 fields Ord_Id, Cust_Id, Cust_name, Ord_date. I like to Store only 2 fields Ord_Id and Cust_Id to Hive. How could I do it? pls advise.

 

below code can't execute:

pig -useHCatalog

orders = LOAD '<HDFS_file_path>' USAGE PigStorage(',') AS (Ord_Id:Int, Cust_Id:Int, Cust_name:chararray, Ord_date:chararray);

STORE orders INTO 'pig_demo.orders' USAGE org.apache.hive.hcatalog.pig.HCatStorer();

 

thanks.

1 REPLY 1
Highlighted

Re: Load 2 fields only out 4 fields - Pig to Hive table

Guru

Hi Alikanti,

 

Have you tried to add the following before you STORE orders:

 

orders2 = FOREACH orders GENERATE Ord_Id, Cust_Id;

 

then

 

STOREorders2 INTO 'pig_demo.orders' USAGE org.apache.hive.hcatalog.pig.HCatStorer();

 

See whether it helps. More information can be found here:

https://pig.apache.org/docs/r0.16.0/start.html#data-work-with

 

Cheers