How can I handle zombie tcp connections?

[#1]

If the client application exits before the impala connection is terminated

Impala hosts have zombie tcp connections.

step 1 : connection to impalad using jdbc

step 2 : query

step 3 : application shutdown without jdbc connection closed

I closed it on the web ui(<impalad host>:25000/sessions ) of the impalad daemon, but the tcp connection does not disappear.

I set 'idle_session_timeout' but tcp connection does not disappear.

[#2]

idle_session_timeout should kill the zombie sessions from the impala server side.

How did you check those zombie connections? How long is your session timeout?

[#3]

My CDH cluster version is cdh6.0.1.

services is kafka, hdfs, hive, impala, hue.

I tested it in the following order.

1. set idele timeout value

   cloudera manager > impala > snippet(safe value) 

   -idle_query_timeout=30

   -idle_session_timeout=120

2. check timeout value.

   cloudera manager > impala > instance > impala daemon web ui > http://[impalad host].example.com:25000/varz

   idle_session_timeout  value setted 120

3. monitoring server's tcp connections

   ssh connect to impalad host

   $ watch -n 1 -d 'netstat -anpt | grep 21050'

4. java client application jdbc 

Connection connection = DriverManager.getconnection("jdbc:impala://....");
Statement statement = connection.createStament();
System.exit(0);

5. impalad server. establishe tcp connection

   The connection will not be terminated after multi minutes.

6. check impala daemon web ui

   can see tcp connection in web ui.

   impala daemon web ui > http://[impalad host].example.com:25000/sessions

   do action 'close'

   However, the tcp connection is still established.

[#4]

My CDH cluster version is cdh6.0.1.

services is kafka, hdfs, hive, impala, hue.

I tested it in the following order.

1. set idele timeout value

   cloudera manager > impala > snippet(safe value)

   -idle_query_timeout=30

   -idle_session_timeout=120

2. check timeout value.

   cloudera manager > impala > instance > impala daemon web ui > http://[impalad host]:25000/varz

   idle_session_timeout value setted 120

3. monitoring server's tcp connections

   ssh connect to impalad host

   $ watch -n 1 -d 'netstat -anpt | grep 21050'

4. java client application jdbc

   Connection connection = DriverManager.getconnection("jdbc:impala://....");
   Statement statement = connection.createStament();
   System.exit(0);
5. impalad server. establishe tcp connection

   The connection will not be terminated after multi minutes.

6. check impala daemon web ui

   can see tcp connection in web ui.

   impala daemon web ui > http://[impalad host]:25000/sessions

do action 'close'

However, the tcp connection is still established.