Created 06-14-2018 08:36 PM
It is running fine sometimes and failing at somepoint with the below error.
Put HiveQL Nifi processor is failing sometimes with Hive Connection pool - Error: Could not establish connection to jdbc:hive2://example.host.com:10001/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;: org.apache.http.client.ClientProtocolException (state=08S01,code=0)
Caused by: org.apache.thrift.transport.TTransportException: org.apache.http.client.ClientProtocolException
        at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:297)
        at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313)
        at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
        at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
        at org.apache.hive.service.cli.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:158)
        at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:150)
        at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:563)
        ... 35 common frames omitted
Caused by: org.apache.http.client.ClientProtocolException: null
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251)
        ... 41 common frames omitted
Caused by: org.apache.http.HttpException: null
        at org.apache.hive.jdbc.HttpRequestInterceptorBase.process(HttpRequestInterceptorBase.java:86)
        at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:183)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:85)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        ... 44 common frames omitted
Caused by: org.apache.http.HttpException: null
        at org.apache.hive.jdbc.HttpKerberosRequestInterceptor.addHttpAuthHeader(HttpKerberosRequestInterceptor.java:68)
        at org.apache.hive.jdbc.HttpRequestInterceptorBase.process(HttpRequestInterceptorBase.java:74)
        ... 50 common frames omitted
Caused by: java.lang.reflect.UndeclaredThrowableException: null
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1887)
        at org.apache.hive.service.auth.HttpAuthUtils.getKerberosServiceTicket(HttpAuthUtils.java:83)
        at org.apache.hive.jdbc.HttpKerberosRequestInterceptor.addHttpAuthHeader(HttpKerberosRequestInterceptor.java:62)
        ... 51 common frames omitted
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
        at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        at org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:183)
        at org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:151)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
					
				
			
			
				
			
			
			
			
			
			
			
		Created 06-14-2018 09:57 PM
It's a bug in NiFi, couldn't able to refresh the kerberos ticket, to fix this issue please upgrade to HDF-3.1.2 (or) get in touch with support team regarding this issue to get Hot fix.
Refer to this link regarding HDF-3.1.2 release
Created 06-14-2018 09:20 PM
Yes it is nifi 1.5
Created 06-14-2018 09:47 PM
Once you disable/enable HiveController service the issue is not going to happen unitil sometime right?
Created 06-14-2018 09:51 PM
Exactly...
Created 06-14-2018 09:21 PM
Which version of NiFi/HDF are you using is it NiFi-1.5?
Created 06-14-2018 09:57 PM
It's a bug in NiFi, couldn't able to refresh the kerberos ticket, to fix this issue please upgrade to HDF-3.1.2 (or) get in touch with support team regarding this issue to get Hot fix.
Refer to this link regarding HDF-3.1.2 release
Created 06-15-2018 07:16 PM
What version of HDF/NiFi are you using? As @Shu mentioned, this is a known issue with HDF 3.1.1. The reason is the Kerberos ticket is not auto-generated once it is expired. Due to which, the connection to Hive is not established and your query doesn't run.
I would recommend upgrading to HDF 3.1.2 to fix this issue, provided you are on a lower HDF version and this is the reason for your problem.
Please have a look at this document for further details on HDF 3.1.2 release and the issues that it is addressing.