Reply
Explorer
Posts: 8
Registered: ‎01-22-2017
Accepted Solution

Is there is any method to read property config file from impala UDF?

[ Edited ]

Is there is any way to read config file from C++ impala UDF ? 

 

It will be  better if we can read the property configured in conf.impala/core-site.xml or conf.impala/hdfs-site.xml from impala UDF.

Cloudera Employee
Posts: 433
Registered: ‎07-29-2015

Re: Is there is any method to read property config file from impala UDF?

Hi Akhil,
It is technically possible to read files from an Impala C++ UDF, since we
don't sandbox UDFs. However, I would strongly recommend against this
because it can lead to resource usage problems and any mistakes can
compromise the stability of Impala.

My recommendation is to rethink your workflow to avoid the need for this
kind of approach if possible.

- Tim
Explorer
Posts: 8
Registered: ‎01-22-2017

Re: Is there is any method to read property config file from impala UDF?

Hi Tim,

 

I want to get the value of fs.defaultFS property from hdfs-site.xml , Impala already have it's value stored somewhere I guess. Is there is any way to get this property value?

Explorer
Posts: 8
Registered: ‎01-22-2017

Re: Is there is any method to read property config file from impala UDF?

Hi,

 

How can I use this piece of code in my impala UDF? what are the libraries and headers I need to include to use this?

 const char* default_fs = ExecEnv::GetInstance()->default_fs().c_str();

 

 

I found this in Impala-cdh5-trunk/be/src/util/hdfs-util.cc , Impala-cdh5-trunk/be/src/runtime/exec-env.cc

 

Or is there is any alternate method to get the value of default_fs ?

 

Cloudera Employee
Posts: 433
Registered: ‎07-29-2015

Re: Is there is any method to read property config file from impala UDF?

We don't support UDFs messing around with Impala's runtime data structures. We don't expose this to UDFs since UDFs aren't really meant to do things like I/O.