question Re: Example for Dynamic Invokers in Pig?? in Archives of Support Questions (Read Only)

Example for Dynamic Invokers in Pig??

bsuresh — Mon, 21 Dec 2015 21:43:17 GMT

I have seen URLDecoder as an example for Pig Dynamic Invokers. Can i have more examples like String related functions or may be other that pig use as dynamic invokers?

Re: Example for Dynamic Invokers in Pig??

aervits — Thu, 24 Dec 2015 12:01:34 GMT

awesome question, didn't even know this exists but digging through Pig source code, here are some examples https://github.com/apache/pig/blob/a44b85a0ab941cd...

-- invoking a static method

DEFINE StringToLong InvokeForLong('java.lang.Long.valueOf', 'String') ;

longs = FOREACH strings GENERATE StringToLong(some_chararray);

-- invoking a method on an object

DEFINE StringConcat InvokeForString('java.lang.String.concat', 'String String', 'false') ;

concatenations = FOREACH strings GENERATE StringConcat(str1, str2);

and here are test cases that you can probably look for examples https://github.com/apache/pig/blob/a44b85a0ab941cd...

Re: Example for Dynamic Invokers in Pig??

bsuresh — Thu, 24 Dec 2015 12:54:07 GMT

pig is awesome...

Re: Example for Dynamic Invokers in Pig??

bsuresh — Thu, 24 Dec 2015 13:11:15 GMT

@Artem Ervits

Is StringConcat function that you defined working for you??? I am encountering following error.

Caused by: java.lang.NoSuchMethodException: java.lang.String.concat() at java.lang.Class.getMethod(Class.java:1665) at org.apache.pig.builtin.Invoker.<init>(Invoker.java:88) at org.apache.pig.builtin.GenericInvoker.<init>(GenericInvoker.java:90) at org.apache.pig.builtin.InvokeForString.<init>(InvokeForString.java:40) ... 39 more

I am not sure whether Pig Dynamic Invokes work for functions other than "Static".

Re: Example for Dynamic Invokers in Pig??

aervits — Thu, 24 Dec 2015 21:58:44 GMT

@Suresh Bonam The string concat example is from the tests of dynamic invokers, I didn't create it. Here are some comments from the source code

/** need more tests -- non-String funcs and especially the full path through the pig interpreter. * I tested manually, seems to work, but * should really add more here.

It's passing their tests so I assume it does work. I will test it at some point. If you want a speedier answer, perhaps you can ping the Pig mailing list? I'm so glad you brought this up, I see how this feature can be really handy and it's been available since Pig 0.8! Here's some more info on it https://squarecog.wordpress.com/?s=dynamic+invoker.

Re: Example for Dynamic Invokers in Pig??

aervits — Fri, 25 Dec 2015 01:50:49 GMT

@Suresh Bonam I tried the StringConcat and it didn't work, even though GenericInvoker does accept functions that are not Static. InvokeForString('java.lang.String.concat', 'String String', 'false') with false being isStatic didn't work for me. I ping the Pig mailing list, will see. It is pretty limiting with only primitives and only static methods, that's a shame!

Re: Example for Dynamic Invokers in Pig??

aervits — Fri, 25 Dec 2015 01:52:26 GMT

Here's the original post https://pig.apache.org/docs/r0.15.0/api/org/apache/pig/builtin/GenericInvoker.html