@zorrofrombrasil How are you running this job? Is it through Zeppelin or any JDBC app? The reason I ask is because I see a keep-alive query "SELECT 1" being run, so the session does not go idle. Nevertheless, this "Invalid SessionHandle" could appear if the connection has switched over to a different hiveserver2 instance. The reason could be a network glitch, or any communication drop(timeout) at the middle layer such as Load Balancer/Knox(if in use). If we are dealing with such large data size, it is better to use a native thrift client like "beeline" that can be run in a Screen so it is not interrupted when a user closes a terminal, or shuts down the client computer. For JDBC application, make sure SocketTimeout is disabled(value = 0), and hive.server2.idle.operation.timeout & hive.server2.idle.session.timeout are set to large values like 6hrs and 8hrs respectively in Hive configuration.
... View more
