Member since
12-02-2019
2
Posts
0
Kudos Received
0
Solutions
01-29-2022
12:51 PM
I have created a .jar file that utilizes many functions from google.i18n.phonenumbers java package. The jar file correctly loads the necessary external libraries from: com.googlecode.libphonenumber.libphonenumber,
com.googlecode.libphonenumber.geocoder,
com.googlecode.libphonenumber.carrier The .jar file is used to create a HIVE UDF function with the following command (on Hue Editor-Cloudera 6.x): create function func_name as 'util.ParserUtil'; The .jar file is located on HIVE_AUX_JARS_PATH=/usr/lib/hive/ for the command above to execute successfully. However, the UDF function func_name is used in a SQL query file located on hdfs. Here is the problem: When the SQL query is executed through Oozie workflow (spawning an execute.sql file) the Oozie throws the below exception: Error: java.lang.RuntimeException: java.lang.NoSuchMethodError: com.google.i18n.phonenumbers.PhoneNumberUtil.parse(Ljava/lang/String;Ljava/lang/String;)Lcom/google/i18n/phonenumbers/Phonenumber$PhoneNumber; , which is awkward since the jar file has both the com.google.i18n.phonenumbers.PhoneNumberUtil.parse() and com/google/i18n/phonenumbers/Phonenumber$PhoneNumber classes (github) When the same SQL query is executed through Hive Editor on Hue or a beeline query it's successfully completed. Thus, in my guess I must apply one of the following two solutions: (this is my current configuration) Can oozie take into consideration the .jar file when it's located explicitly at HIVE_AUX_JARS_PATH=/usr/lib/hive/?, when running the workflow. Or shall I upload the .jar file (that creates the UDF and includes the necessary java classes from google.i18n) on hdfs path /user/oozie/share/lib/hive/ Or upload .jar file on a lib folder on hdfs like /user/src/myFile.jar and provide that path on oozie workflow .xml file like <file>hdfs://user/src/myFile.jar</file>? As a result, the oozie workflow will execute correctly the hive udf function from myFile.jar. Does point (1) suffice or should I apply one of the points (2), (3)?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Oozie