Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 03-03-2017 08:27 AM
Hi, I am started to use the Sandbox and I am quite confused how to register my Java UDF in Pig. I tried to put the jar file into the HDFS and in the docker image too, but I always get the same error: ERROR 101: file 'myudf.jar' does not exist.
Should I put the file in the cent os file system? And if the answer is yes, than how can I do it?
Thanks,
Zsoka
Created 03-03-2017 03:23 PM
https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.
REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar'; DEFINE SIMPLEUPPER exampleudf.UPPER(); typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray); upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row); DUMP upper_typing_line;
Good luck & happy Hadooping!
Created 03-03-2017 11:44 AM
Here's an example I'm using piggybank UDF library, jar is on local filesystem
REGISTER /usr/hdp/current/pig-client/piggybank.jar;
A = LOAD 'input' USING PigStorage();
STORE A INTO 'output' USING org.apache.pig.piggybank.storage.MultiStorage('output', '0');
Created 03-03-2017 03:23 PM
https://martin.atlassian.net/wiki/x/C4BRAQ shows an example (and old one that I did back with Hue) of loading the jar file into HDFS and then registering it like in the below example.
REGISTER 'hdfs:///user/hue/shared/pig/udfs/exampleudf.jar'; DEFINE SIMPLEUPPER exampleudf.UPPER(); typing_line = LOAD '/user/hue/testData/typingText.txt' AS (row:chararray); upper_typing_line = FOREACH typing_line GENERATE SIMPLEUPPER(row); DUMP upper_typing_line;
Good luck & happy Hadooping!
Created 04-10-2017 01:43 PM
Looks like a couple of good answers. Please check out https://martin.atlassian.net/wiki/x/AunyBQ and if one of these answer helps you the "best", would you mind "Accepting" it??