Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on
01-29-2020
08:35 AM
- last edited on
01-29-2020
11:35 AM
by
ask_bill_brooks
Hi all,
I'm trying to create a function to use in imapla.
my function is simply re-using hive's sha2() function.
the creation of the function goes smoothly
:
create function to_sha2(string,int) returns string location 'user/hive/hive.jar' symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2' ;
but when I try to use it doesn't work launching this warning :
select to_sha2('test',256);
Query State: EXCEPTION
Query Status: ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2
I have tried to search for UDFSha2 that doesn't contain the Generic word in the hive's jar but I couldn't find it.
the original built-in function in hive :
sha2(string/binary, len) - Calculates the SHA-2 family of hash functions (SHA-224, SHA-256, SHA-384, and SHA-512). |
the other functions are working normally in impala (for example I have tried to create UDF MD5 function from hive's jar and it was working ).
so my question is do I have to create my own UDF-Sha2 function?
or there is a saving situation for my case, any help will be appreciated
Impala version 2.9
Hive : 1.1.0
CDH : 5.12
Created on 01-30-2020 08:10 AM - edited 01-30-2020 08:11 AM
After some researches I did, it seems that Impala does not support GenericUDFs yet.
https://issues.apache.org/jira/browse/IMPALA-7877
https://issues.apache.org/jira/browse/IMPALA-8369
so I'll just try to create my own function for Impala.
Created on 01-30-2020 08:10 AM - edited 01-30-2020 08:11 AM
After some researches I did, it seems that Impala does not support GenericUDFs yet.
https://issues.apache.org/jira/browse/IMPALA-7877
https://issues.apache.org/jira/browse/IMPALA-8369
so I'll just try to create my own function for Impala.
Created 03-18-2020 08:36 PM
Hi,
Do you have any results of your own sh function in Impala?
Regards
Rosa
Created 03-24-2020 07:17 AM
HI Rosa,
Sorry I gave up that time because It was an urgent matter , so I just took the short way and used hive to hash my data and put it in a table , where I can run my queries later with Impala ,
I'll come back for it later for sure since hive is a bit slow while having java based functions.
I'd recommend you to try with C language ,it's suitable for impala tho it will work faster .
So please If you came up with anything share it with us , otherwise I'll post for sure my solution once it's done .
best luck
Bilal
Created 03-24-2020 08:08 AM
Hi!
thanks for your response and good luck too! 🙂
Rosita