Created 07-08-2016 08:34 PM
hive> add jar urldetector-1.0-jar-with-dependencies.jar; add jar urldetector-1.0-jar-with-dependencies.jar Added [urldetector-1.0-jar-with-dependencies.jar] to class path Added resources: [urldetector-1.0-jar-with-dependencies.jar] hive> CREATE TEMPORARY FUNCTION urldetector as 'com.dataflowdeveloper.detection.URLDetector'; CREATE TEMPORARY FUNCTION urldetector as 'com.dataflowdeveloper.detection.URLDetector' OK Time taken: 0.032 seconds hive> select urldetector(remote_host), remote_host from AccessLogs limit 100; select urldetector(remote_host), remote_host from AccessLogs limit 100 OK _c0 remote_host Sent: 68.180.229.154 Failed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method public org.apache.hadoop.io.Text com.dataflowdeveloper.detection.URLDetector.evaluate(org.apache.hadoop.io.Text) throws java.io.IOException on object com.dataflowdeveloper.detection.URLDetector@5dbd1a3e of class com.dataflowdeveloper.detection.URLDetector with arguments {68.180.229.154:org.apache.hadoop.io.Text} of size 1 Time taken: 0.147 seconds
This is when trying to load a Hive UDF.
Created 07-08-2016 09:50 PM
So the issue was the library I was using was compiled with JDK 8 and everything else is JDK 7. There was no issue listed, JUnit and local Java applications ran fine. When I manually uploaded the JAR, it gave me the dreaded "Unsupported major.minor version 52.0"
With a properly compiled library, we will be fine.
So make sure you compile in JDK 7 if your Hadoop / Hive platform is JDK 7
Created 07-08-2016 09:50 PM
So the issue was the library I was using was compiled with JDK 8 and everything else is JDK 7. There was no issue listed, JUnit and local Java applications ran fine. When I manually uploaded the JAR, it gave me the dreaded "Unsupported major.minor version 52.0"
With a properly compiled library, we will be fine.
So make sure you compile in JDK 7 if your Hadoop / Hive platform is JDK 7