question Re: IMPALA function 'hive generic UDF SHA2 ' in Support Questions

IMPALA function 'hive generic UDF SHA2 '

Bildervic — Wed, 29 Jan 2020 19:35:13 GMT

Hi all,
I'm trying to create a function to use in imapla.
my function is simply re-using hive's sha2() function.
the creation of the function goes smoothly
:
create function to_sha2(string,int) returns string location 'user/hive/hive.jar' symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2' ;

but when I try to use it doesn't work launching this warning :

select to_sha2('test',256);

 Query State: EXCEPTION
    Query Status: ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2

I have tried to search for UDFSha2 that doesn't contain the Generic word in the hive's jar but I couldn't find it.
the original built-in function in hive :
sha2(string/binary, len) - Calculates the SHA-2 family of hash functions (SHA-224, SHA-256, SHA-384, and SHA-512). |
the other functions are working normally in impala (for example I have tried to create UDF MD5 function from hive's jar and it was working ).

so my question is do I have to create my own UDF-Sha2 function?
or there is a saving situation for my case, any help will be appreciated

Impala version 2.9
Hive : 1.1.0
CDH : 5.12

Re: IMPALA function 'hive generic UDF SHA2 '

Bildervic — Thu, 30 Jan 2020 16:11:36 GMT

After some researches I did, it seems that Impala does not support GenericUDFs yet.
https://issues.apache.org/jira/browse/IMPALA-7877

https://issues.apache.org/jira/browse/IMPALA-8369

so I'll just try to create my own function for Impala.

Re: IMPALA function 'hive generic UDF SHA2 '

RosaT — Thu, 19 Mar 2020 03:36:20 GMT

Hi,

Do you have any results of your own sh function in Impala?

Regards

Rosa

Re: IMPALA function 'hive generic UDF SHA2 '

Bildervic — Tue, 24 Mar 2020 14:17:36 GMT

HI Rosa,

Sorry I gave up that time because It was an urgent matter , so I just took the short way and used hive to hash my data and put it in a table , where I can run my queries later with Impala ,

I'll come back for it later for sure since hive is a bit slow while having java based functions.
I'd recommend you to try with C language ,it's suitable for impala tho it will work faster .

So please If you came up with anything share it with us , otherwise I'll post for sure my solution once it's done .

best luck
Bilal

Re: IMPALA function 'hive generic UDF SHA2 '

RosaT — Tue, 24 Mar 2020 15:08:18 GMT

Hi!

thanks for your response and good luck too! 🙂

Rosita