question Re: Hive UDF not running with other user than HDFS in Archives of Support Questions (Read Only)

Hive UDF not running with other user than HDFS

rogerio_biondi — Sat, 15 Oct 2016 06:47:35 GMT

Hello,

I'm using the latest hortonworks release, plain installation and just created a simple UDF function for Hive:

import org.apache.hadoop.hive.ql.exec.Description;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.DoubleWritable;
@Description (
	name="test",
	value="returns the sum of two double values",
	extended="SELECT test(a, b) from foo limit 1;"
)

public class Test extends UDF {
	public DoubleWritable evaluate(DoubleWritable a, DoubleWritable b) {
        	return new DoubleWritable(a.get() + b.get());
	}
}

Then created the jar and uploaded to server.

Added the jar to hive and created the temporary function:

add jar hdfs:/user/myuser/Test-0.0.1-SNAPSHOT.jar;
create temporary function test as 'Test';

*** When I run this function with the "hdfs" user, it works PERFECT.

But when I switch to another user (myuser) and do the same procedure above, receive the following error when running the UDF:

Error while compiling statement: FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments '46.71': No matching method for class Test with (double, double). Possible choices:

Any idea why only hdfs user can run this UDF? Thank you very much.

Best regards,

Rogerio Biondi

Re: Hive UDF not running with other user than HDFS

grajagopal — Sat, 15 Oct 2016 23:36:18 GMT

@Rogerio Biondi

What is the permission set on the UDF jar file ?

Re: Hive UDF not running with other user than HDFS

rogerio_biondi — Mon, 17 Oct 2016 19:29:10 GMT

Changed to 777 but the result is the same...

Re: Hive UDF not running with other user than HDFS

cstanca — Wed, 19 Oct 2016 09:09:55 GMT

@Rogerio Biondi

You added it only for the context of the user that you used when created the temporary function. If you want the function to be available for any user then create it as a permanent function (remove "temporary"):

add jar hdfs:/user/myuser/Test-0.0.1-SNAPSHOT.jar;

create function test as'Test';

This function will be available to all users until you restart Hive2 Server.

+++

Pls vote and accept best answer, if any.

Re: Hive UDF not running with other user than HDFS

robson_ximenes — Wed, 28 Feb 2018 02:29:54 GMT

Your function is expecting two double parameters, your example is sending only one.