A customer recently asked about the security capabilities of HIVE with regards to masking and encrypting specific fields within a HIVE table. After some research, I found that there are 4 such UDFs already available in the HIVE 1.3 release. Unfortunately, the customer was using HDP 2.3.0, which ships with HIVE 0.14, and thus they did NOT have access to these UDFs. https://cwiki.apache.org/confluence/display/Hive/L... Being the impatient type, and not wanting to reinvent the wheel, I created a simple github project to back-port these UDFs into, which can be found here: https://github.com/davidkj69/Backported-UDFs Then, I harvested the code from the Apache HIVE 1.3 trunk and added it to the repo, changed the library versions to Hive 0.14, and ran mvn build to generate the Backported-UDFs-0.0.1.jar, which I loaded onto HDFS in the /user/davidk directory. I then was able to test all of the functions as follows: hive> add jar Backported-UDFs-0.0.1.jar; hive> CREATE function sha1 AS 'org.apache.hadoop.hive.ql.udf.UDFSha1' USING JAR 'hdfs:///user/davidk/Backported-UDFs-0.0.1.jar'; hive> select sha1('ABC'); hive> CREATE function aes_encrypt AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFAesEncrypt' USING JAR 'hdfs:///user/davidk/Backported-UDFs-0.0.1.jar'; hive> select base64(aes_encrypt('ABC', '1234567890123456')); hive> CREATE function aes_decrypt AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFAesDecrypt' USING JAR 'hdfs:///user/davidk/Backported-UDFs-0.0.1.jar'; hive> select aes_decrypt(unbase64('y6Ss+zCYObpCbgfWfyNWTw=='), '1234567890123456'); hive> CREATE function sha2 AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFSha2' USING JAR 'hdfs:///user/davidk/Backported-UDFs-0.0.1.jar'; hive> select sha2('ABC', 256); If you or your customers don't want to wait to leverage these UDFs, go to my repo, build the jar, and start using them right away!! Happ Hadooping
... View more
