Skip to main content
FantaCola
Bronze III
November 29, 2024
Question

Connect Apache NiFi opensource version 1.24.0 to CDP 7.1.7 Hive 3.1.3

Hello everyone. Hope you are doing well.
I am trying to achieve to run simple query on hive, using nifi processors. I Have tested couple scenarios, with different versions (also tried apache kyuubi 1.8.0 with jdbc drivers).
CDP private cloud 7.1.7

Environment
Apache NiFi - Version 1.18.0 (java 11)
Hive - Version 3.1.3

jdbc driver is downloaded from cloudera, version 2.6.25 and using HiveJDBC41.jar

1. scenario
using Apache NiFi DBCPConnectionPool 1.24.0
URL : jdbc:hive2://example.com:10000/default;KrbRealm=EXAMPLE.COM;KrbHostFQDN=example.com;KrbServiceName=hive;AuthMech=1;KrbHostFQDN=_HOST;principal=test@EXAMPLE.COM
Driver Class : com.cloudera.hive.jdbc41.HS2Driver
Driver Location : /opt/nifi/nifi-current/lib/DRIVER.jar
and using kerberos credential service with principal/keytab

Nifi error log same as on screenshot.

2. scenario
using HiveConnectionPool 1.19.1
url : jdbc:hive2://example.com:10000/default
Hive Configuration Resources : /opt/nifi/nifi-current/conf/hive/hive-site.xml,/opt/nifi/nifi-current/conf/hadoop/core-site.xml
and using kerberos credential service with principal/keytab

I have also tested lower and latest version of Apache Nifi but same errors. Tried also Apache Kyuubi in middle and using its shaded bundle jar for jdbc but same errors. Mby someone has any experience with this. 

Regards


Capture.PNG

1 reply

DianaTorres
Community Manager
Community Manager
November 29, 2024

Hi! @MattWho @ggangadharan do you have any insights here? Thanks!

Translate
Regards, Diana Torres, Senior Community Moderator