Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Is there is any method to read property config file from impala UDF?

avatar
Explorer

Is there is any way to read config file from C++ impala UDF ? 

 

It will be  better if we can read the property configured in conf.impala/core-site.xml or conf.impala/hdfs-site.xml from impala UDF.

1 ACCEPTED SOLUTION

avatar

We don't support UDFs messing around with Impala's runtime data structures. We don't expose this to UDFs since UDFs aren't really meant to do things like I/O.

View solution in original post

4 REPLIES 4

avatar
Hi Akhil,
It is technically possible to read files from an Impala C++ UDF, since we
don't sandbox UDFs. However, I would strongly recommend against this
because it can lead to resource usage problems and any mistakes can
compromise the stability of Impala.

My recommendation is to rethink your workflow to avoid the need for this
kind of approach if possible.

- Tim

avatar
Explorer

Hi Tim,

 

I want to get the value of fs.defaultFS property from hdfs-site.xml , Impala already have it's value stored somewhere I guess. Is there is any way to get this property value?

avatar
Explorer

Hi,

 

How can I use this piece of code in my impala UDF? what are the libraries and headers I need to include to use this?

 const char* default_fs = ExecEnv::GetInstance()->default_fs().c_str();

 

 

I found this in Impala-cdh5-trunk/be/src/util/hdfs-util.cc , Impala-cdh5-trunk/be/src/runtime/exec-env.cc

 

Or is there is any alternate method to get the value of default_fs ?

 

avatar

We don't support UDFs messing around with Impala's runtime data structures. We don't expose this to UDFs since UDFs aren't really meant to do things like I/O.