NiFi Hive Connection Pool Error.

[#1]

Hi,

I am trying to connect to create a connection Hive using different formats DB Connection URLs. All of them are giving different errors.

first DB Connection URL and Error:

jdbc:hive2://MyServer:10000/hdf_moat;principal=hive/MyServer@MyDomain;

error:

21:37:29 UTCERRORd3b62ee6-0157-1000-b66f-364970fcfa98 SelectHiveQL[id=d3b62ee6-0157-1000-b66f-364970fcfa98] Unable to execute HiveQL select query show tables due to org.apache.nifi.processor.exception.ProcessException: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport with JDBC Uri: jdbc:hive2://MyServer:10000/hdf_moat;principal=hive/MyServer@MyDomain: GSS initiate failed). No FlowFile to route to failure: org.apache.nifi.processor.exception.ProcessException: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport with JDBC Uri: jdbc:hive2://MyServer:10000/hdf_moat;principal=hive/MyServer@MyDomain: GSS initiate failed)

Second DB Connection URL and Error:

jdbc:hive2://MyServer:10000/hdf_moat;principal=hive/MyServer@MyDomain;transportMode=http;httpPath=cliservice

Error

21:47:09 UTCERRORd3b62ee6-0157-1000-b66f-364970fcfa98 SelectHiveQL[id=d3b62ee6-0157-1000-b66f-364970fcfa98] Unable to execute HiveQL select query show tables due to org.apache.nifi.processor.exception.ProcessException: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of class 'org.apache.hive.jdbc.HiveDriver' for connect URL ' jdbc:hive2://MyServer:10000/hdf_moat;principal=hive/MyServer@MyDomain;transportMode=http;httpPath=cliservice'. No FlowFile to route to failure: org.apache.nifi.processor.exception.ProcessException: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of class 'org.apache.hive.jdbc.HiveDriver' for connect URL ' jdbc:hive2://MyServer:10000/hdf_moat;principal=hive/MyServer@MyDomain;transportMode=http;httpPath=cliservice'

Third DB Connection and Error

jdbc:hive2://MyServer:10000/hdf_moat;principal=hive/MyServer@MyDomain;transportMode=http;httpPath=cliservice;auth=KERBEROS

Error 21:49:56 UTCERRORd3b62ee6-0157-1000-b66f-364970fcfa98 SelectHiveQL[id=d3b62ee6-0157-1000-b66f-364970fcfa98] SelectHiveQL[id=d3b62ee6-0157-1000-b66f-364970fcfa98] failed to process session due to java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory: java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory 21:49:56 UTCWARNINGd3b62ee6-0157-1000-b66f-364970fcfa98 SelectHiveQL[id=d3b62ee6-0157-1000-b66f-364970fcfa98]

Processor Administratively Yielded for 1 sec due to processing failure

for all of these above i am commonly getting a warning

HiveConnectionPool[id=d3b68593-0157-1000-2894-db6013bf04f3] Configuration does not have security enabled, Keytab and Principal will be ignored

Thanks in advance,

Sai

[#2]

@Saikrishna Tarapareddy

GSS initiate failed exception simply means that the provided credentials are incorrect.

You must have kerberised the cluster with a principal/keytab combination. Please provide that as the kerberos principa;/keytab. The credentials that you have provided are that of the hive service.

[#3]

@mkalyanpur

our NiFi is running as NiFi service account with NiFi principal user , Hive is running as Hive user. I tried with both and getting the same error.

is that what you are asking.? sorry for the confusion.

[#4]

@Saikrishna Tarapareddy Were you able to resolve this issue. I am having similar issue. Please let me know.

[#5]

Could you also add core-site.xml in Hive Configuration Properties from your kerberosed cluster please?

[#6]

I was having the same issue but after I added core-site.xml alongside my hive-site.xml file I was able to connect. Thanks!

/usr/hdp/current/hive/conf/hive-site.xml,/etc/hadoop/current/0/core-site.xml

[#7]

@mkalyanpur do I have to copy the krb5.conf file from my Hive server to the NiFi server?