hadoop-core and hive-exec versions for UDF

[#1]

Hello,

How should I choose hadoop-core and hive-exec jar versions for Impala UDFs? What should they match in the target CDH installation?

For instance, hive-exec is now at version 3.1.0 but most examples I've seen use 0.13.x...

Thank you,

Gerasimos

[#2]

I'd recommend using the versions matching your CDH installation.

[#3]

That is what I am asking. I have CDH 5.11, which reports:

- hadoop-2.6.0+cdh5.11.2+2429

- hive-1.1.0+cdh5.11.2+1082

Does this mean that I have to use hadoop-core-2.6.0 ? There isn't such version, as the max is hadoop-core-1.2.1.

[#4 · Accepted solution]

We have a CDH Maven repository with the CDH version of things. CDH releases don't always align exactly with Apache releases - they may include additional bug fixes, features or may have non-production-ready features disabled.

I believe these are the docs for how to reference them via maven:

https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh5_maven_repo.html

E.g. here is hadoop-core for 5.11.2: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hadoop/hadoop-core/2.6.0-mr1-c...

[#5]

Thank you,

I added cloudera repo to my gradle configuration and builds and functions run from Impala without problems.