Reply
Highlighted
New Contributor
Posts: 5
Registered: ‎01-08-2018

Apache Impala UDF session

Hello,

 

I'm facing an issue with Apache Impala UDF.

 

Background -

I'm using CDH 5.13 -

Apache Hive ver. - hive-1.1.0
Apache Impala ver. - impala.2.10.0


I implemented UDF in Java for Hive and Impala to perform the following -

 

1) set_username (string)
    This method keeps the username in SessionState of the UDF for using of calaulate function.
    conf.set(hiveConfKeyName,username);
    SessionState s_state = new SessionState (conf);

    SessionState.setCurrentSessionState(s_state);


2) calculate (int)

    This method gets as an input a number and return true/false according to the username kept in SessionState.

 

In Hive, it's working fine !

 

However, In Impala UDF it's partially working -

 

Impala Good scenario -

When the calculate function is being called with const value, I can fetch the username from sessionstate succesfully

 

example -

 

create view my_view as select

t.name,

CASE WHEN calculate (1000)

then t.id

else -1

END as ID.

t.address

from test_table t

 

 

Impala Bad scenario -

 

When the calculate function is being called with dynamic value, the SessionState is not kept, therefore, I can not fetch the username from sessionstate

 

example -

 

create view my_view as select

t.name,

CASE WHEN calculate (t.id)

then t.id

else -1

END as ID.

t.address

from test_table t

 

I noticed in the logs of the "bad scenario" that thread-id for set_username and calculate is differant.

In good example, it's same thread id.

 

Again, in Hive UDF it's working just fine with the view presented above.

 

 

 

Is it a bug/feature ?

Any idea hoe to overcome this issue ?

 

 

Thank you

Announcements