Support Questions
Find answers, ask questions, and share your expertise

IMPALA function 'hive generic UDF SHA2 '

Solved Go to solution

IMPALA function 'hive generic UDF SHA2 '

Contributor

Hi all,
I'm trying to create a function to use in imapla.
my function is simply re-using  hive's sha2() function.
the creation of the function goes smoothly 
:
create function to_sha2(string,int) returns string location 'user/hive/hive.jar' symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2' ; 


but when I try to use it doesn't work launching this warning  : 

select to_sha2('test',256);

 

 Query State: EXCEPTION
    Query Status: ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2

 

 

I have tried to search for UDFSha2 that doesn't contain the Generic word in the hive's jar but I couldn't find it.
the original built-in function in hive :
sha2(string/binary, len) - Calculates the SHA-2 family of hash functions (SHA-224, SHA-256, SHA-384, and SHA-512). |
the other functions are working normally in impala (for example  I have tried to create UDF MD5 function from hive's jar and it was working ).

so my question is do I have to create my own UDF-Sha2  function?
or there is a saving situation for my case, any help will be appreciated 




Impala version 2.9
Hive : 1.1.0
CDH : 5.12

1 ACCEPTED SOLUTION

Accepted Solutions

Re: IMPALA function 'hive generic UDF SHA2 '

Contributor

After some researches I did, it seems that Impala does not support GenericUDFs yet. 
https://issues.apache.org/jira/browse/IMPALA-7877

https://issues.apache.org/jira/browse/IMPALA-8369

so I'll just try to create my own function for Impala.

View solution in original post

4 REPLIES 4

Re: IMPALA function 'hive generic UDF SHA2 '

Contributor

After some researches I did, it seems that Impala does not support GenericUDFs yet. 
https://issues.apache.org/jira/browse/IMPALA-7877

https://issues.apache.org/jira/browse/IMPALA-8369

so I'll just try to create my own function for Impala.

View solution in original post

Re: IMPALA function 'hive generic UDF SHA2 '

New Contributor

Hi,

 

Do you have any results of your own sh function in Impala? 

 

Regards

Rosa

Re: IMPALA function 'hive generic UDF SHA2 '

Contributor

HI Rosa,

 

Sorry I gave up that time  because It was an urgent matter  , so I just took the short way and used hive to hash my data and put it in a table , where I can run my queries later with Impala  , 

I'll come back for it later for sure since hive is a bit slow  while having java based functions.
I'd recommend you to try with C language  ,it's suitable for impala tho it will work faster .

So please If you came up with anything share it with us , otherwise I'll post for sure my solution once it's done .

best luck
Bilal

Re: IMPALA function 'hive generic UDF SHA2 '

New Contributor

Hi! 

thanks for your response and good luck too!  

 

Rosita