Support Questions

Find answers, ask questions, and share your expertise

Oozie workflow fails to execute hive query using UDF with com.google.i18n.phonenumbers.PhoneNumberUtil.parse()

New Contributor

I have created a .jar file that utilizes many functions from google.i18n.phonenumbers java package. The jar file correctly loads the necessary external libraries from:

 

com.googlecode.libphonenumber.libphonenumber,
com.googlecode.libphonenumber.geocoder,
com.googlecode.libphonenumber.carrier

 

The .jar file is used to create a HIVE UDF function with the following command (on Hue Editor-Cloudera 6.x):

 

create function func_name as 'util.ParserUtil';

 

The .jar file is located on HIVE_AUX_JARS_PATH=/usr/lib/hive/ for the command above to execute successfully.

 

However, the UDF function func_name is used in a SQL query file located on hdfs. Here is the problem:

  • When the SQL query is executed through Oozie workflow (spawning an execute.sql file) the Oozie throws the below exception:

 

Error: java.lang.RuntimeException: java.lang.NoSuchMethodError: com.google.i18n.phonenumbers.PhoneNumberUtil.parse(Ljava/lang/String;Ljava/lang/String;)Lcom/google/i18n/phonenumbers/Phonenumber$PhoneNumber;​


, which is awkward since the jar file has both the com.google.i18n.phonenumbers.PhoneNumberUtil.parse() and com/google/i18n/phonenumbers/Phonenumber$PhoneNumber classes (github)

 

  • When the same SQL query is executed through Hive Editor on Hue or a beeline query it's successfully completed.

Thus, in my guess I must apply one of the following two solutions:

  1. (this is my current configuration) Can oozie take into consideration the .jar file when it's located explicitly at HIVE_AUX_JARS_PATH=/usr/lib/hive/?, when running the workflow.
  2. Or shall I upload the .jar file (that creates the UDF and includes the necessary java classes from google.i18n) on hdfs path /user/oozie/share/lib/hive/
  3. Or upload .jar file on a lib folder on hdfs like /user/src/myFile.jar and provide that path on oozie workflow .xml file like <file>hdfs://user/src/myFile.jar</file>? As a result, the oozie workflow will execute correctly the hive udf function from myFile.jar.

Does point (1) suffice or should I apply one of the points (2), (3)?

0 REPLIES 0
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.