I'm installing a Hive UDF (in Java) on Impala with the HDFS location of the JAR but whenever Impala is restarted the UDFs are missing. I'm using CDH 5.3.2 with Impala 2.1.2.
Please let me know if there is anyway to not loose the UDF or trigger the install after every restart.
I'm afraid the UDFs are lost after restarting the catalogd. One way to avoid the issue is to only selectively restart the impalad and/or statestored if that is suffucient in your setup.
Btw, we use https://issues.cloudera.org/browse/IMPALA-1748 to track the feature of persistent UDFs.
I'm also running into this issue with Java UDFs stored in HDFS. The current Cloudera docs however state that this should work:
Currently, Impala UDFs and UDAs written in C++ are not persisted in the metastore database. Information about these functions is held in the memory of thecatalogd daemon. You must reload them by running the CREATE FUNCTION statements again each time you restart the catalogd daemon. This limitation does not apply to Impala UDFs and UDAs written in Java.
Are the docs incorrect in this case or am I doing something stupid?
Any updates on the above issue.
I have created hive udf and using with impala to query hbase.
i am also facing the same issue. Udfs are getting dropped after restarting daemons.
We have around 50 udfs which needs to be created after scheduled every cluster restart or impala service restart.
I cant see any progress in https://issues.cloudera.org/browse/IMPALA-1748.
Is there any temporary solution to avoid create udfs again and again after restarting ?
I am also facing similar issue, as we restart impala service the UDF's are gone and we have to recreate them manually or through a script, Is there any fix or a way to avoid manually creating them after restart as it is becoming an important and complicated issue for us. Please help me out with this. Also is there any way to add them permanently.
We're tracking the issue with https://issues.cloudera.org/browse/IMPALA-1748. We're aware that a number of people are hitting this issue and take that into account when prioritizing it relative to other bugfixes and features.
Today I installed CDH quickstart vm 5.5 which has Impala 2.3.0+cdh5.5.0+0, But still I am seeing same issue, after restart of Impala service the UDF's are gone and again I have to manually create it.
Can you please tell me when will be this fixed and what will that CDH and Impala release be. Its very urgent issue, as our code is sometimes failing in production.
There is a patch for this under review now. There is a release scheduled for sometime around April-May next year and the patch should be included.