Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Register UDF in PIG

avatar
New Contributor

Hi, I am started to use the Sandbox and I am quite confused how to register my Java UDF in Pig. I tried to put the jar file into the HDFS and in the docker image too, but I always get the same error: ERROR 101: file 'myudf.jar' does not exist.

Should I put the file in the cent os file system? And if the answer is yes, than how can I do it?

Thanks,

Zsoka

1 ACCEPTED SOLUTION

avatar

https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.

REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar';
DEFINE SIMPLEUPPER exampleudf.UPPER();
typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray);
upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row);
DUMP upper_typing_line;

Good luck & happy Hadooping!

View solution in original post

3 REPLIES 3

avatar
Master Mentor

Here's an example I'm using piggybank UDF library, jar is on local filesystem

REGISTER /usr/hdp/current/pig-client/piggybank.jar;

A = LOAD 'input' USING PigStorage();
STORE A INTO 'output' USING org.apache.pig.piggybank.storage.MultiStorage('output', '0');

avatar

https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.

REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar';
DEFINE SIMPLEUPPER exampleudf.UPPER();
typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray);
upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row);
DUMP upper_typing_line;

Good luck & happy Hadooping!

avatar

Looks like a couple of good answers. Please check out https://martin.atlassian.net/wiki/x/AunyBQ and if one of these answer helps you the "best", would you mind "Accepting" it??