question hadoop-core and hive-exec versions for UDF in Support Questions

hadoop-core and hive-exec versions for UDF

gerasimos — Fri, 16 Sep 2022 13:36:39 GMT

Hello,

How should I choose hadoop-core and hive-exec jar versions for Impala UDFs? What should they match in the target CDH installation?

For instance, hive-exec is now at version 3.1.0 but most examples I've seen use 0.13.x...

Thank you,

Gerasimos

Re: hadoop-core and hive-exec versions for UDF

Tim Armstrong — Tue, 21 Aug 2018 18:15:14 GMT

I'd recommend using the versions matching your CDH installation.

Re: hadoop-core and hive-exec versions for UDF

gerasimos — Wed, 22 Aug 2018 06:57:49 GMT

That is what I am asking. I have CDH 5.11, which reports:

- hadoop-2.6.0+cdh5.11.2+2429

- hive-1.1.0+cdh5.11.2+1082

Does this mean that I have to use hadoop-core-2.6.0 ? There isn't such version, as the max is hadoop-core-1.2.1.

Re: hadoop-core and hive-exec versions for UDF

Tim Armstrong — Thu, 23 Aug 2018 16:51:12 GMT

We have a CDH Maven repository with the CDH version of things. CDH releases don't always align exactly with Apache releases - they may include additional bug fixes, features or may have non-production-ready features disabled.

I believe these are the docs for how to reference them via maven:

https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh5_maven_repo.html

E.g. here is hadoop-core for 5.11.2: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hadoop/hadoop-core/2.6.0-mr1-cdh5.11.2/

Re: hadoop-core and hive-exec versions for UDF

gerasimos — Fri, 24 Aug 2018 07:42:13 GMT

Thank you,

I added cloudera repo to my gradle configuration and builds and functions run from Impala without problems.