Created 03-03-2017 08:27 AM
Hi, I am started to use the Sandbox and I am quite confused how to register my Java UDF in Pig. I tried to put the jar file into the HDFS and in the docker image too, but I always get the same error: ERROR 101: file 'myudf.jar' does not exist.
Should I put the file in the cent os file system? And if the answer is yes, than how can I do it?
Thanks,
Zsoka
Created 03-03-2017 03:23 PM
https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.
REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar'; DEFINE SIMPLEUPPER exampleudf.UPPER(); typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray); upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row); DUMP upper_typing_line;
Good luck & happy Hadooping!
Created 03-03-2017 11:44 AM
Here's an example I'm using piggybank UDF library, jar is on local filesystem
REGISTER /usr/hdp/current/pig-client/piggybank.jar; A = LOAD 'input' USING PigStorage(); STORE A INTO 'output' USING org.apache.pig.piggybank.storage.MultiStorage('output', '0');
Created 03-03-2017 03:23 PM
https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.
REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar'; DEFINE SIMPLEUPPER exampleudf.UPPER(); typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray); upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row); DUMP upper_typing_line;
Good luck & happy Hadooping!
Created 04-10-2017 01:43 PM
Looks like a couple of good answers. Please check out https://martin.atlassian.net/wiki/x/AunyBQ and if one of these answer helps you the "best", would you mind "Accepting" it??