question Impala UDF C++ : risks to deploy in a production environnement in Archives of Support Questions (Read Only)

Impala UDF C++ : risks to deploy in a production environnement

Plop564 — Fri, 16 Sep 2022 12:37:15 GMT

Hello,

We have multiples Impala C++ UDFs that we want to deploy on our production Cloudera cluster.

We have carefully rewieved the source code, in order to avoid memory leaks, segmentation fault and race conditions.

However, if we have not seen something and a segmentation fault, memory leaks, or race conditions still occurs, what could be the risks for the entire cluster ?

If an error like that occurs, does the corresponding Impalad could crash ? If an Impalad crash owing to an UDF, restarting it will be enough to go back to good health ?

What about the Impalad isolation ? Again if a segmentation fault, memory leaks or race conditions occurs, does other cloudera services instances can be affected ? (HDFS, Hive, ...)

Could you please quickly summarize the associated risks with buggy C++ UDF ?

Thanks !

Re: Impala UDF C++ : risks to deploy in a production environnement

Tim Armstrong — Fri, 08 Dec 2017 20:03:06 GMT

Hi @Plop564

> If an error like that occurs, does the corresponding Impalad could crash ? If an Impalad crash owing to an UDF, restarting it will be enough to go back to good health ?

Yes and yes.

>What about the Impalad isolation ? Again if a segmentation fault, memory leaks or race conditions occurs, does other cloudera services instances can be affected ? (HDFS, Hive, ...)

It won't affect services outside of Impala - the crash is isolated to the Impala process

> Could you please quickly summarize the associated risks with buggy C++ UDF ?

You already mentioned the possibilities of crashes, memory leaks and memory corruption. The other thing to keep in mind is that the UDF runs within the Impala process so essentially has the same permissions as the "impala" user. A malicious UDF could exploit this. This is why we recommend reviewing UDF code. It sounds like you're already following that best practice.

Re: Impala UDF C++ : risks to deploy in a production environnement

Plop564 — Sun, 10 Dec 2017 19:15:55 GMT

Hi @Tim Armstrong

Thank you very much for the reply !