Created on 01-29-2020 08:35 AM - last edited on 01-29-2020 11:35 AM by ask_bill_brooks
Hi all,
I'm trying to create a function to use in imapla.
my function is simply re-using hive's sha2() function.
the creation of the function goes smoothly
:
create function to_sha2(string,int) returns string location 'user/hive/hive.jar' symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2' ;
but when I try to use it doesn't work launching this warning :
select to_sha2('test',256);
Query State: EXCEPTION Query Status: ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2
I have tried to search for UDFSha2 that doesn't contain the Generic word in the hive's jar but I couldn't find it.
the original built-in function in hive :
sha2(string/binary, len) - Calculates the SHA-2 family of hash functions (SHA-224, SHA-256, SHA-384, and SHA-512). |
the other functions are working normally in impala (for example I have tried to create UDF MD5 function from hive's jar and it was working ).
so my question is do I have to create my own UDF-Sha2 function?
or there is a saving situation for my case, any help will be appreciated
Impala version 2.9
Hive : 1.1.0
CDH : 5.12
Created on 01-30-2020 08:10 AM - edited 01-30-2020 08:11 AM
After some researches I did, it seems that Impala does not support GenericUDFs yet.
https://issues.apache.org/jira/browse/IMPALA-7877
https://issues.apache.org/jira/browse/IMPALA-8369
so I'll just try to create my own function for Impala.
Created on 01-30-2020 08:10 AM - edited 01-30-2020 08:11 AM
After some researches I did, it seems that Impala does not support GenericUDFs yet.
https://issues.apache.org/jira/browse/IMPALA-7877
https://issues.apache.org/jira/browse/IMPALA-8369
so I'll just try to create my own function for Impala.
Created 03-18-2020 08:36 PM
Hi,
Do you have any results of your own sh function in Impala?
Regards
Rosa
Created 03-24-2020 07:17 AM
HI Rosa,
Sorry I gave up that time because It was an urgent matter , so I just took the short way and used hive to hash my data and put it in a table , where I can run my queries later with Impala ,
I'll come back for it later for sure since hive is a bit slow while having java based functions.
I'd recommend you to try with C language ,it's suitable for impala tho it will work faster .
So please If you came up with anything share it with us , otherwise I'll post for sure my solution once it's done .
best luck
Bilal
Created 03-24-2020 08:08 AM
Hi!
thanks for your response and good luck too! 🙂
Rosita