Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there is any method to read property config file from impala UDF?

SOLVED Go to solution

Is there is any method to read property config file from impala UDF?

Explorer

Is there is any way to read config file from C++ impala UDF ? 

 

It will be  better if we can read the property configured in conf.impala/core-site.xml or conf.impala/hdfs-site.xml from impala UDF.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Is there is any method to read property config file from impala UDF?

Master Collaborator

We don't support UDFs messing around with Impala's runtime data structures. We don't expose this to UDFs since UDFs aren't really meant to do things like I/O.

4 REPLIES 4

Re: Is there is any method to read property config file from impala UDF?

Master Collaborator
Hi Akhil,
It is technically possible to read files from an Impala C++ UDF, since we
don't sandbox UDFs. However, I would strongly recommend against this
because it can lead to resource usage problems and any mistakes can
compromise the stability of Impala.

My recommendation is to rethink your workflow to avoid the need for this
kind of approach if possible.

- Tim

Re: Is there is any method to read property config file from impala UDF?

Explorer

Hi Tim,

 

I want to get the value of fs.defaultFS property from hdfs-site.xml , Impala already have it's value stored somewhere I guess. Is there is any way to get this property value?

Re: Is there is any method to read property config file from impala UDF?

Explorer

Hi,

 

How can I use this piece of code in my impala UDF? what are the libraries and headers I need to include to use this?

 const char* default_fs = ExecEnv::GetInstance()->default_fs().c_str();

 

 

I found this in Impala-cdh5-trunk/be/src/util/hdfs-util.cc , Impala-cdh5-trunk/be/src/runtime/exec-env.cc

 

Or is there is any alternate method to get the value of default_fs ?

 

Re: Is there is any method to read property config file from impala UDF?

Master Collaborator

We don't support UDFs messing around with Impala's runtime data structures. We don't expose this to UDFs since UDFs aren't really meant to do things like I/O.