Support Questions

Find answers, ask questions, and share your expertise

IMPALA function 'hive generic UDF SHA2 '

avatar
Rising Star

Hi all,
I'm trying to create a function to use in imapla.
my function is simply re-using  hive's sha2() function.
the creation of the function goes smoothly 
:
create function to_sha2(string,int) returns string location 'user/hive/hive.jar' symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2' ; 


but when I try to use it doesn't work launching this warning  : 

select to_sha2('test',256);

 

 Query State: EXCEPTION
    Query Status: ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2

 

 

I have tried to search for UDFSha2 that doesn't contain the Generic word in the hive's jar but I couldn't find it.
the original built-in function in hive :
sha2(string/binary, len) - Calculates the SHA-2 family of hash functions (SHA-224, SHA-256, SHA-384, and SHA-512). |
the other functions are working normally in impala (for example  I have tried to create UDF MD5 function from hive's jar and it was working ).

so my question is do I have to create my own UDF-Sha2  function?
or there is a saving situation for my case, any help will be appreciated 




Impala version 2.9
Hive : 1.1.0
CDH : 5.12

1 ACCEPTED SOLUTION

avatar
Rising Star

After some researches I did, it seems that Impala does not support GenericUDFs yet. 
https://issues.apache.org/jira/browse/IMPALA-7877

https://issues.apache.org/jira/browse/IMPALA-8369

so I'll just try to create my own function for Impala.

View solution in original post

4 REPLIES 4

avatar
Rising Star

After some researches I did, it seems that Impala does not support GenericUDFs yet. 
https://issues.apache.org/jira/browse/IMPALA-7877

https://issues.apache.org/jira/browse/IMPALA-8369

so I'll just try to create my own function for Impala.

avatar
Explorer

Hi,

 

Do you have any results of your own sh function in Impala? 

 

Regards

Rosa

avatar
Rising Star

HI Rosa,

 

Sorry I gave up that time  because It was an urgent matter  , so I just took the short way and used hive to hash my data and put it in a table , where I can run my queries later with Impala  , 

I'll come back for it later for sure since hive is a bit slow  while having java based functions.
I'd recommend you to try with C language  ,it's suitable for impala tho it will work faster .

So please If you came up with anything share it with us , otherwise I'll post for sure my solution once it's done .

best luck
Bilal

avatar
Explorer

Hi! 

thanks for your response and good luck too! 🙂 

 

Rosita