Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

Wildfly data source with Kerberos security domain and Impala JDBC driver can't recycle connections

Highlighted

Wildfly data source with Kerberos security domain and Impala JDBC driver can't recycle connections

Explorer

In our product, we use Wildfly 10.1.0 and configure a data source with Kerberos security domain and Impala JDBC driver to connect to a Kerberos(1.15.1-19) enabled Impalad instance(CDH 5.16.1). In basic test, it can connect successfully. But after navigating from pages to pages for several times, it will hang without response.

 

In Cloudera Manager console, I can see the critical warning:

  • The health test result for IMPALAD_FRONTEND_CONNECTIONS has become bad: There are 0 (Beeswax pool) 62 (Hive Server 2 pool) active client connections, each pool has a configured maximum of 64.

 

In Wildfly console, when monitoring the data source statistics, I can see that the number of active conntections is always increasing and can exceed the configured max number of connections 28. When it reaches 64, the system will hang.

 

It seems like a connection leak. But if I remove the Kerberos security domain configuration and connect to an unsecured impala instance, it works well(no code changes). So it has nothing to do with our codes.

 

Does anyone have the same experience as me? How to solved this problem?

 

My data source configuration of wildfly is as follows.

<security-domain name="kerberos-impala">
<authentication>
<login-module code="org.jboss.security.negotiation.KerberosLoginModule" flag="required" module="org.jboss.security.negotiation" >
<module-option name="storeKey" value="true"/>
<module-option name="useKeyTab" value="true"/>
<module-option name="keyTab" value="/opt/ossa_st/ossa.keytab"/>
<module-option name="principal" value="ossa/osscdh01.gre.hpecorp.net@OSSCDH01.GRE.HPECORP.NET"/>
<module-option name="useTicketCache" value="false"/>
<module-option name="debug" value="true"/>
<module-option name="refreshKrb5Config" value="true"/>
<module-option name="isInitiator" value="true"/>
<module-option name="doNotPrompt" value="true"/>
<module-option name="addGSSCredential" value="true"/>
</login-module>
</authentication>
</security-domain>

... ...

<datasource jta="false" jndi-name="java:jboss/datasources/OssaFaultDS" pool-name="OssaFaultDS" enabled="true" statistics-enabled="true"> <connection-url>jdbc:impala://osscdh02.gre.hpecorp.net:21050/fas_st;AuthMech=1;KrbHostFQDN=osscdh02.gre.hpecorp.net;KrbServiceName=impala</connection-url> <driver>impala</driver> <pool> <max-pool-size>28</max-pool-size> </pool> <security> <security-domain>kerberos-impala</security-domain> </security> <validation> <check-valid-connection-sql>SELECT 1</check-valid-connection-sql> <validate-on-match>false</validate-on-match> <background-validation>true</background-validation> <background-validation-millis>120000</background-validation-millis> </validation> <timeout> <blocking-timeout-millis>300000</blocking-timeout-millis> </timeout> </datasource> <drivers> <driver name="impala" module="com.cloudera.impala"> <driver-class>com.cloudera.impala.jdbc41.Driver</driver-class> </driver> </drivers>

 

 

1 REPLY 1

Re: Wildfly data source with Kerberos security domain and Impala JDBC driver can't recycle connectio

Explorer