Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Register UDF in PIG

avatar
New Member

Hi, I am started to use the Sandbox and I am quite confused how to register my Java UDF in Pig. I tried to put the jar file into the HDFS and in the docker image too, but I always get the same error: ERROR 101: file 'myudf.jar' does not exist.

Should I put the file in the cent os file system? And if the answer is yes, than how can I do it?

Thanks,

Zsoka

1 ACCEPTED SOLUTION

avatar

https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.

REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar';
DEFINE SIMPLEUPPER exampleudf.UPPER();
typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray);
upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row);
DUMP upper_typing_line;

Good luck & happy Hadooping!

View solution in original post

3 REPLIES 3

avatar
Master Mentor

Here's an example I'm using piggybank UDF library, jar is on local filesystem

REGISTER /usr/hdp/current/pig-client/piggybank.jar;

A = LOAD 'input' USING PigStorage();
STORE A INTO 'output' USING org.apache.pig.piggybank.storage.MultiStorage('output', '0');

avatar

https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.

REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar';
DEFINE SIMPLEUPPER exampleudf.UPPER();
typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray);
upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row);
DUMP upper_typing_line;

Good luck & happy Hadooping!

avatar

Looks like a couple of good answers. Please check out https://martin.atlassian.net/wiki/x/AunyBQ and if one of these answer helps you the "best", would you mind "Accepting" it??