Support Questions
Find answers, ask questions, and share your expertise

Python CDSW Impala.dbapi.connect connection doesn't close

Python CDSW Impala.dbapi.connect connection doesn't close

New Contributor

Following the documentation here to setup and close a connection in python cdsw via impala.dbapi, I've found that the connection seems to remain open as I'm able to continue to use it for a query after I've run

connection.close()

What needs to be done to close the connection? The cluster admin insists that even when the session closes, the connections are remaining open and taking up resources.

 

here's the test:

 

 

 

#Python 2
from impala.dbapi import connect
import pandas as pd

#### Set up impala connection in conn_imp
conn_imp = connect(host='phxhadoopp08.swift.com',port = 21050,auth_mechanism = 'GSSAPI')

#### Read sql from conn_imp
df = pd.read_sql('Select * FROM prod_ba.dart_pred_hst Limit 10',conn_imp)
print df.shape

#### Close Connection
conn_imp.close()

#### Read sql from conn_imp to see if closed
df = pd.read_sql('Select * FROM prod_ba.dart_pred_hst Limit 10',conn_imp)
print df.shape

 

 

 

and here's a screen shot of the results:

 

image.png

1 REPLY 1

Re: Python CDSW Impala.dbapi.connect connection doesn't close

Master Collaborator

@chris_snedaker Can you help me to understand what is the difference between if below two are similar in nature? 

connection.close()

conn_imp.close()

I think you should try with connection.close() and see if this works. Also is this the same behaviour outside of CDSW if you test the same code from host. This can help you to narrow down the situation. 


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.