keep alive - Error message from Server: Invalid SessionHandle: SessionHandle

[#1]

I run the database "ASK_SELL" with 43 millions of records with size of 122 GB in Hadoop. Its run for almost 1 hour and 40 minutes and stop it with the error message bellow.

ERROR: Execute error:[Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code:0, SQL state: null, Query: SELECT 1 -- /* keep alive */, Error message from Server: Invalid SessionHandle: SessionHandle
WARNING: File deletion failed for DESTINO2.BENEF_CONC.DATA.

Whats going on ?

[#2]

@zorrofrombrasil Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Hive experts @ggangadharan @james_jones @Shmoo  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

• The error message Invalid SessionHandle: SessionHandle commonly occurs in Hive when there is an issue with the session handle being used.
• A session handle in Hive is a unique identifier for a session created when a user connects to Hive, used to maintain the state and context of the session.
• One possible scenario for this error is when a table contains a large number of records and the cluster has multiple HS2 instances. If Knox is used to connect to Hive, Knox might connect to one HS2 and run a query. However, due to the large number of records, the query takes longer to process. If the connection times out from Knox's end and reconnects to another HS2, the query might fail with the "Invalid SessionHandle" error.
• To investigate this scenario, it is recommended to check the HS2 logs and Knox logs. Additionally, to determine why the query is running long, checking the HS2 and appLogs for any yarn job initiated by HS2 can provide further insights.

[#4]

@zorrofrombrasil How are you running this job? Is it through Zeppelin or any JDBC app? The reason I ask is because I see a keep-alive query "SELECT 1" being run, so the session does not go idle.

Nevertheless, this "Invalid SessionHandle" could appear if the connection has switched over to a different hiveserver2 instance. The reason could be a network glitch, or any communication drop(timeout) at the middle layer such as Load Balancer/Knox(if in use). 

If we are dealing with such large data size, it is better to use a native thrift client like "beeline" that can be run in a Screen so it is not interrupted when a user closes a terminal, or shuts down the client computer.  For JDBC application, make sure SocketTimeout is disabled(value = 0), and hive.server2.idle.operation.timeout & hive.server2.idle.session.timeout are set to large values like 6hrs and 8hrs respectively in Hive configuration.

[#5]

@zorrofrombrasil 

Did you get a chance to check the updates from @ggangadharan and @smruti?

If so, did the information help to fix the issue? or Do you have any other concerns?