Can pig load data into HAWQ?

[#1]

Can pig load data into HAWQ?

[#2 · Accepted solution]

great question man, apparently yes http://pivotal-field-engineering.github.io/pmr-common/pmr/apidocs/index.html?com/gopivotal/pig/HawqL...

[#3]

straight out of the docs:

SET pig.splitCombination false; A = LOAD 'hawq://mdw1:5432/gpadmin' USING com.gopivotal.pig.HawqLoader('retail_demo.products_dim', 'gpadmin'); DESCRIBE A; A: {product_id: int,category_id: int,price: bigdecimal,product_name: chararray} B = LOAD 'hawq://mdw1:5432/gpadmin' USING com.gopivotal.pig.HawqLoader('retail_demo.vinyl_counts', 'gpadmin'); DESCRIBE B; B: {category_id: int,count: int} C = JOIN A BY category_id, B by category_id USING 'replicated'; STORE C INTO '$output';

[#4]

With PXF, you can do anything. Not sure PIG + HAWQ is the best combo though

[#5]

@Timothy Spann @Artem Ervits I got confirmation Pig CAN NOT load directly into HAWQ. it is not supported.

[#6]

Not supported and not possible is two different things. Docs have a clear example.