question Re: Is there is any method to read property config file from impala UDF? in Support Questions

Is there is any method to read property config file from impala UDF?

AkhilTM — Fri, 16 Sep 2022 10:58:51 GMT

Is there is any way to read config file from C++ impala UDF ?

It will be better if we can read the property configured in conf.impala/core-site.xml or conf.impala/hdfs-site.xml from impala UDF.

Re: Is there is any method to read property config file from impala UDF?

Tim Armstrong — Mon, 30 Jan 2017 18:31:53 GMT

Hi Akhil,
It is technically possible to read files from an Impala C++ UDF, since we
don't sandbox UDFs. However, I would strongly recommend against this
because it can lead to resource usage problems and any mistakes can
compromise the stability of Impala.

My recommendation is to rethink your workflow to avoid the need for this
kind of approach if possible.

- Tim

Re: Is there is any method to read property config file from impala UDF?

AkhilTM — Mon, 30 Jan 2017 19:12:12 GMT

Hi Tim,

I want to get the value of fs.defaultFS property from hdfs-site.xml , Impala already have it's value stored somewhere I guess. Is there is any way to get this property value?

Re: Is there is any method to read property config file from impala UDF?

AkhilTM — Tue, 31 Jan 2017 12:04:03 GMT

Hi,

How can I use this piece of code in my impala UDF? what are the libraries and headers I need to include to use this?

 const char* default_fs = ExecEnv::GetInstance()->default_fs().c_str();

I found this in Impala-cdh5-trunk/be/src/util/hdfs-util.cc , Impala-cdh5-trunk/be/src/runtime/exec-env.cc

Or is there is any alternate method to get the value of default_fs ?

Re: Is there is any method to read property config file from impala UDF?

Tim Armstrong — Wed, 01 Feb 2017 01:25:15 GMT

We don't support UDFs messing around with Impala's runtime data structures. We don't expose this to UDFs since UDFs aren't really meant to do things like I/O.