Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Register UDF in PIG

avatar
New Member

Hi, I am started to use the Sandbox and I am quite confused how to register my Java UDF in Pig. I tried to put the jar file into the HDFS and in the docker image too, but I always get the same error: ERROR 101: file 'myudf.jar' does not exist.

Should I put the file in the cent os file system? And if the answer is yes, than how can I do it?

Thanks,

Zsoka

1 ACCEPTED SOLUTION

avatar

https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.

REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar';
DEFINE SIMPLEUPPER exampleudf.UPPER();
typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray);
upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row);
DUMP upper_typing_line;

Good luck & happy Hadooping!

View solution in original post

3 REPLIES 3

avatar
Master Mentor

Here's an example I'm using piggybank UDF library, jar is on local filesystem

REGISTER /usr/hdp/current/pig-client/piggybank.jar;

A = LOAD 'input' USING PigStorage();
STORE A INTO 'output' USING org.apache.pig.piggybank.storage.MultiStorage('output', '0');

avatar

https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.

REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar';
DEFINE SIMPLEUPPER exampleudf.UPPER();
typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray);
upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row);
DUMP upper_typing_line;

Good luck & happy Hadooping!

avatar

Looks like a couple of good answers. Please check out https://martin.atlassian.net/wiki/x/AunyBQ and if one of these answer helps you the "best", would you mind "Accepting" it??