Application where Hive execute statement is issued:   2021-06-30 07:26:21.637 [SimpleAsyncTaskExecutor-1] DEBUG org.apache.hive.jdbc.HiveStatement - Progress Bar Response Received: TProgressUpdateResp(headerNames:[], rows:[], progressedPercentage:0.0, status:NOT_AVAILABLE, footerSummary:, startTime:0) 2021-06-30 07:26:21.637 [SimpleAsyncTaskExecutor-1] DEBUG org.apache.thrift.transport.TSaslTransport - data length before wrap: 104 2021-06-30 07:26:21.637 [SimpleAsyncTaskExecutor-1] DEBUG org.apache.thrift.transport.TSaslTransport - writing data length: 164 2021-06-30 07:28:21.667 [SimpleAsyncTaskExecutor-1] DEBUG org.apache.thrift.transport.TSaslTransport - CLIENT: reading data length: 169 2021-06-30 07:28:21.667 [SimpleAsyncTaskExecutor-1] DEBUG org.apache.thrift.transport.TSaslTransport - data length after unwrap: 109 2021-06-30 07:28:21.667 [SimpleAsyncTaskExecutor-1] DEBUG org.apache.hive.jdbc.HiveStatement - Progress Bar Response Received: TProgressUpdateResp(headerNames:[], rows:[], progressedPercentage:0.0, status:NOT_AVAILABLE, footerSummary:, startTime:0) 2021-06-30 07:32:10.945 [SimpleAsyncTaskExecutor-1] DEBUG org.springframework.batch.core.step.tasklet.TaskletStep - Saving step execution before commit: StepExecution: id=235168, version=1, name=BaseHiveExec-914344549, status=STARTED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription= 2021-06-30 07:32:10.945 [SimpleAsyncTaskExecutor-1] DEBUG org.springframework.orm.jpa.JpaTransactionManager - Found thread-bound EntityManager [SessionImpl(656304976<open>)] for JPA transaction 2021-06-30 07:32:10.945 [SimpleAsyncTaskExecutor-1] DEBUG org.springframework.orm.jpa.JpaTransactionManager - Participating in existing transaction 2021-06-30 07:32:10.962 [SimpleAsyncTaskExecutor-1] DEBUG org.springframework.batch.core.job.flow.support.SimpleFlow - Completed state=BasePipelineFLow-1-1046646724.BaseHiveExec-914344549 with status=COMPLETED       3. Version HDP : 2.6    Hive jars being used:  hive-jdbc-1.2.1000.2.6.5.0-292-sources.jar,  hive-service-1.2.1000.2.6.5.0-292.jar   @asish Any insights to the issue would be helpful. Thanks ! ... View more

1. HiveServer2Logs   2021-06-27 00:00:10,298 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64374, dagName=FROM feed.PhysioData_FE...patientTimeMapping(Stage-10), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=hive_20210626235952_39a8be69-15c6-4ff9-b2af-bd0b391eee42 } 2021-06-27 00:00:10,309 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClientUtils (TezClientUtils.java:getAMProxy(850)) - Application not running, applicationId=application_1622134593468_64374, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64374/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 2021-06-27 00:00:10,309 INFO [HiveServer2-Background-Pool: Thread-27753154]: exec.Task (SessionState.java:printInfo(1106)) - Tez session was closed. Reopening... 2021-06-27 00:00:10,309 INFO [HiveServer2-Background-Pool: Thread-27753154]: tez.TezSessionPoolManager (TezSessionPoolManager.java:close(226)) - Closing tez session default? false 2021-06-27 00:00:10,309 INFO [HiveServer2-Background-Pool: Thread-27753154]: tez.TezSessionState (TezSessionState.java:close(294)) - Closing Tez Session 2021-06-27 00:00:10,309 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:stop(518)) - Shutting down Tez Session, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64374 2021-06-27 00:00:10,309 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClientUtils (TezClientUtils.java:getAMProxy(850)) - Application not running, applicationId=application_1622134593468_64374, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64374/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 2021-06-27 00:00:10,309 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:stop(532)) - Failed to shutdown Tez Session via proxy org.apache.tez.dag.api.SessionNotRunning: Application not running, applicationId=application_1622134593468_64374, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64374/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 at org.apache.tez.client.TezClientUtils.getAMProxy(TezClientUtils.java:851) at org.apache.tez.client.TezClient.getAMProxy(TezClient.java:736) at org.apache.tez.client.TezClient.stop(TezClient.java:524) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.close(TezSessionState.java:296) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.close(TezSessionPoolManager.java:228) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeAndOpen(TezSessionPoolManager.java:353) at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:503) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:196) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:177) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:96) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1777) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1511) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1308) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1175) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76) at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:273) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2021-06-27 00:00:10,310 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:stop(537)) - Could not connect to AM, killing session via YARN, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64374 2021-06-27 00:00:10,310 INFO [HiveServer2-Background-Pool: Thread-27753154]: impl.YarnClientImpl (YarnClientImpl.java:killApplication(417)) - Killed application application_1622134593468_64374 2021-06-27 00:00:16,415 INFO [HiveServer2-Background-Pool: Thread-27747067]: session.SessionState (SessionState.java:getHdfsEncryptionShim(465)) - Could not get hdfsEncryptionShim, it is only applicabl e to hdfs filesystem. 2021-06-27 00:00:16,415 INFO [HiveServer2-Background-Pool: Thread-27747067]: session.SessionState (SessionState.java:getHdfsEncryptionShim(465)) - Could not get hdfsEncryptionShim, it is only applicabl e to hdfs filesystem. 2021-06-27 00:00:16,462 INFO [HiveServer2-Background-Pool: Thread-27709561]: fs.FSStatsPublisher (FSStatsPublisher.java:init(52)) - created : s3a://cf-s3-3b4640e1-bc89-4021-b54d-3400fee16fad/fz/enumera tionattribute/master/.hive-staging_hive_2021-06-26_23-48-03_151_254699988323216319-81386/-ext-10005 2021-06-27 00:00:16,476 INFO [HiveServer2-Background-Pool: Thread-27751248]: monitoring.TezJobMonitor$UpdateFunction (TezJobMonitor.java:update(137)) - Map 1: 0(+1)/1 2021-06-27 00:00:16,481 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64424, dagName=TezPreWarmDAG_0 2021-06-27 00:00:16,494 INFO [HiveServer2-Handler-Pool: Thread-12797611]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(784)) - 680567: get_table : db=master tbl=SourceDevice 2021-06-27 00:00:16,494 INFO [HiveServer2-Handler-Pool: Thread-12797611]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=pcmsdas ip=unknown-ip-addr cmd=get_table : db=master tbl=SourceDevice 2021-06-27 00:00:16,496 INFO [HiveServer2-Handler-Pool: Thread-17672971]: orc.OrcUtils (OrcUtils.java:getDesiredRowTypeDescr(810)) - Using schema evolution configuration variables schema.evolution.columns [id, firstname, lastname, middlename, dateofbirth, gender, displayname, primaryid, alias, alternateid, encounterid, lifetimeid, transportlocation, height, weight, conflictstate, pacedstate, category, admitstate, resuscitationstatus, heightunit, weightunit, pressureunit, ts, createdat, bedlabel, clinicalunit, timezone, institutionname, collectorid, tenantname] / schema.evolution.columns.types [string, string, string, string, timestamp, int, string, string, string, string, string, string, string, float, float, int, int, int, int, int, int, int, int, timestamp, timestamp, string, string, string, string, string, string] (isAcid false) 2021-06-27 00:00:16,584 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:submitDAGSession(501)) - Submitted dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64424, dagName=TezPreWarmDAG_0 2021-06-27 00:00:16,586 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=serializePlan start=1624752016584 end=1624752016586 duration=2 from=org.apache.hadoop.hive.ql.exec.Utilities> 2021-06-27 00:00:16,587 INFO [HiveServer2-Background-Pool: Thread-27753623]: exec.Utilities (Utilities.java:setBaseWork(726)) - Setting plan: /tmp/hive/pcmsdas/77381f1b-3b93-4169-adf2-a391c6d52a92/hive_2021-06-27_00-00-13_834_2638864708697550412-101110/46d0c3a0-6154-4ebf-9944-2093044bd409/map.xml 2021-06-27 00:00:16,593 INFO [HiveServer2-Background-Pool: Thread-27753623]: exec.Task (TezTask.java:build(420)) - Setting tez.task.scale.memory.reserve-fraction to 0.30000001192092896 2021-06-27 00:00:16,594 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=TezCreateVertex.Map 1 start=1624752016582 end=1624752016594 duration=12 from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2021-06-27 00:00:16,596 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogBegin(149)) - <PERFLOG method=TezCreateVertex.Map 3 from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2021-06-27 00:00:16,596 INFO [HiveServer2-Background-Pool: Thread-27753623]: ql.Context (Context.java:getMRScratchDir(476)) - New scratch dir is hdfs://**:8020/tmp/hive/pcmsdas/77381f1b-3b93-4169-adf2-a391c6d52a92/hive_2021-06-27_00-00-13_834_2638864708697550412-101110 2021-06-27 00:00:16,598 INFO [HiveServer2-Background-Pool: Thread-27753623]: tez.DagUtils (DagUtils.java:createVertex(557)) - Vertex has custom input? false 2021-06-27 00:00:16,598 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogBegin(149)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities> 2021-06-27 00:00:16,598 INFO [HiveServer2-Background-Pool: Thread-27753623]: exec.Utilities (Utilities.java:serializePlan(1040)) - Serializing MapWork via kryo 2021-06-27 00:00:16,599 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=serializePlan start=1624752016598 end=1624752016599 duration=1 from=org.apache.hadoop.hive.ql.exec.Utilities> 2021-06-27 00:00:16,599 INFO [HiveServer2-Background-Pool: Thread-27753623]: exec.Utilities (Utilities.java:setBaseWork(726)) - Setting plan: /tmp/hive/pcmsdas/77381f1b-3b93-4169-adf2-a391c6d52a92/hive_2021-06-27_00-00-13_834_2638864708697550412-101110/2dba0a2a-4791-4fed-837f-5e75bc07790c/map.xml 2021-06-27 00:00:16,605 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=TezCreateVertex.Map 3 start=1624752016596 end=1624752016605 duration=9 from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2021-06-27 00:00:16,607 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=TezBuildDag start=1624752016048 end=1624752016607 duration=559 from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2021-06-27 00:00:16,607 INFO [HiveServer2-Background-Pool: Thread-27753623]: log.PerfLogger (PerfLogger.java:PerfLogBegin(149)) - <PERFLOG method=TezSubmitDag from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2021-06-27 00:00:16,608 INFO [HiveServer2-Background-Pool: Thread-27753623]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-cac1797f-8ea2-4f70-b954-ffe568685a56, applicationId=application_1622134593468_64422, dagName=insert into table `ma...imarykey1,tenantname(Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=hive_20210627000013_e61fa8f3-3a65-4cb5-9bbe-fcd077a0d276 } 2021-06-27 00:00:22,272 INFO [HiveServer2-Background-Pool: Thread-27753154]: exec.Task (SessionState.java:printInfo(1106)) - Session re-established. 2021-06-27 00:00:22,273 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64424, dagName=FROM feed.PhysioData_FE...patientTimeMapping(Stage-10), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=hive_20210626235952_39a8be69-15c6-4ff9-b2af-bd0b391eee42 } 2021-06-27 00:00:22,318 INFO [HiveServer2-Background-Pool: Thread-27753840]: impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(302)) - Timeline service address: http://**:8188/ws/v1/timeline/ 2021-06-27 00:00:22,318 INFO [HiveServer2-Background-Pool: Thread-27753154]: client.TezClient (TezClient.java:submitDAGSession(501)) - Submitted dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64424, dagName=FROM feed.PhysioData_FE...patientTimeMapping(Stage-10) 2021-06-27 00:00:22,327 INFO [HiveServer2-Background-Pool: Thread-27709561]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(784)) - 996584: get_partition_with_auth : db=master tbl=wavesegment[PHP_Test,2021,6,26,20] 2021-06-27 00:00:22,327 INFO [HiveServer2-Background-Pool: Thread-27709561]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=pcmsdas ip=unknown-ip-addr cmd=get_partition_with_auth : db=master tbl=wavesegment[PHP_Test,2021,6,26,20] 2021-06-27 00:14:59,212 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64424, dagName=FROM feed.PhysioData_FEED_4fe...sourceDevice(Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=hive_20210627001457_c5228bce-f94d-4357-b78b-536daedeb494 } 2021-06-27 00:14:59,222 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClientUtils (TezClientUtils.java:getAMProxy(850)) - Application not running, applicationId=application_1622134593468_64424, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64424/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 2021-06-27 00:14:59,222 INFO [HiveServer2-Background-Pool: Thread-27798033]: exec.Task (SessionState.java:printInfo(1106)) - Tez session was closed. Reopening... 2021-06-27 00:14:59,222 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.TezSessionPoolManager (TezSessionPoolManager.java:close(226)) - Closing tez session default? false 2021-06-27 00:14:59,222 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.TezSessionState (TezSessionState.java:close(294)) - Closing Tez Session 2021-06-27 00:14:59,222 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:stop(518)) - Shutting down Tez Session, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64424 2021-06-27 00:14:59,223 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClientUtils (TezClientUtils.java:getAMProxy(850)) - Application not running, applicationId=application_1622134593468_64424, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64424/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 2021-06-27 00:14:59,223 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:stop(532)) - Failed to shutdown Tez Session via proxy org.apache.tez.dag.api.SessionNotRunning: Application not running, applicationId=application_1622134593468_64424, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64424/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 at org.apache.tez.client.TezClientUtils.getAMProxy(TezClientUtils.java:851) at org.apache.tez.client.TezClient.getAMProxy(TezClient.java:736) at org.apache.tez.client.TezClient.stop(TezClient.java:524) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.close(TezSessionState.java:296) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.close(TezSessionPoolManager.java:228) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeAndOpen(TezSessionPoolManager.java:353) at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:503) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:196) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:177) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:96) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1777) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1511) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1308) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1175) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76) at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:273) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2021-06-27 00:14:59,223 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:stop(537)) - Could not connect to AM, killing session via YARN, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64424 2021-06-27 00:14:59,224 INFO [HiveServer2-Background-Pool: Thread-27798033]: impl.YarnClientImpl (YarnClientImpl.java:killApplication(417)) - Killed application application_1622134593468_64424 2021-06-27 00:14:59,224 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.TezSessionState (TezSessionState.java:open(129)) - Opening the session with id d4bec027-6c39-4549-86dd-305da87a612d for thread HiveServer2-Background-Pool: Thread-27798033 log trace id - query id - hive_20210627001457_c5228bce-f94d-4357-b78b-536daedeb494 2021-06-27 00:14:59,224 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.TezSessionState (TezSessionState.java:open(145)) - User of session id d4bec027-6c39-4549-86dd-305da87a612d is pcmsdas 2021-06-27 00:14:59,225 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.DagUtils (DagUtils.java:createLocalResource(725)) - Resource modification time: 1624668636459 2021-06-27 00:14:59,226 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.DagUtils (DagUtils.java:createLocalResource(725)) - Resource modification time: 1624668636466 2021-06-27 00:14:59,227 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.DagUtils (DagUtils.java:createLocalResource(725)) - Resource modification time: 1624668636475 2021-06-27 00:14:59,228 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.DagUtils (DagUtils.java:createLocalResource(725)) - Resource modification time: 1624668636466 2021-06-27 00:14:59,229 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.DagUtils (DagUtils.java:createLocalResource(725)) - Resource modification time: 1624668636475 2021-06-27 00:14:59,230 INFO [HiveServer2-Background-Pool: Thread-27798033]: tez.DagUtils (DagUtils.java:getHiveJarDirectory(899)) - Jar dir is null/directory doesn't exist. Choosing HIVE_INSTALL_DIR - /user/pcmsdas/.hiveJars 2021-06-27 00:14:59,234 INFO [HiveServer2-Handler-Pool: Thread-27392563]: session.SessionState (SessionState.java:getHdfsEncryptionShim(465)) - Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2021-06-27 00:15:04,891 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64467, dagName=TezPreWarmDAG_0 2021-06-27 00:15:04,974 INFO [HiveServer2-Handler-Pool: Thread-15774433]: orc.OrcUtils (OrcUtils.java:getDesiredRowTypeDescr(810)) - Using schema evolution configuration variables schema.evolution.columns [id, firstname, lastname, middlename, dateofbirth, gender, displayname, primaryid, alias, alternateid, encounterid, lifetimeid, transportlocation, height, weight, conflictstate, pacedstate, category, admitstate, resuscitationstatus, heightunit, weightunit, pressureunit, ts, createdat, bedlabel, clinicalunit, timezone, institutionname, collectorid, tenantname] / schema.evolution.columns.types [string, string, string, string, timestamp, int, string, string, string, string, string, string, string, float, float, int, int, int, int, int, int, int, int, timestamp, timestamp, string, string, string, string, string, string] (isAcid false) 2021-06-27 00:15:04,997 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:submitDAGSession(501)) - Submitted dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64467, dagName=TezPreWarmDAG_0 2021-06-27 00:15:05,001 INFO [HiveServer2-Background-Pool: Thread-27762486]: ql.Driver (Driver.java:launchTask(1770)) - Starting task [Stage-42:STATS] in serial mode 2021-06-27 00:15:05,001 INFO [HiveServer2-Background-Pool: Thread-27762486]: exec.StatsTask (StatsTask.java:execute(90)) - Executing stats task 2021-06-27 00:15:05,001 INFO [HiveServer2-Background-Pool: Thread-27762486]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(784)) - 997550: get_table : db=master tbl=waveattribute 2021-06-27 00:15:05,001 INFO [HiveServer2-Background-Pool: Thread-27762486]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(319)) - ugi=pcmsdas ip=unknown-ip-addr cmd=get_table : db=master tbl=waveattribute 2021-06-27 00:15:11,129 INFO [HiveServer2-Background-Pool: Thread-27798033]: exec.Task (SessionState.java:printInfo(1106)) - Session re-established. 2021-06-27 00:15:11,129 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64467, dagName=FROM feed.PhysioData_FEED_4fe...sourceDevice(Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=hive_20210627001457_c5228bce-f94d-4357-b78b-536daedeb494 } 2021-06-27 00:15:11,165 INFO [HiveServer2-Handler-Pool: Thread-17672971]: orc.OrcUtils (OrcUtils.java:getDesiredRowTypeDescr(810)) - Using schema evolution configuration variables schema.evolution.columns [id, firstname, lastname, middlename, dateofbirth, gender, displayname, primaryid, alias, alternateid, encounterid, lifetimeid, transportlocation, height, weight, conflictstate, pacedstate, category, admitstate, resuscitationstatus, heightunit, weightunit, pressureunit, ts, createdat, bedlabel, clinicalunit, timezone, institutionname, collectorid, tenantname] / schema.evolution.columns.types [string, string, string, string, timestamp, int, string, string, string, string, string, string, string, float, float, int, int, int, int, int, int, int, int, timestamp, timestamp, string, string, string, string, string, string] (isAcid false) 2021-06-27 00:15:11,165 INFO [HiveServer2-Handler-Pool: Thread-17672971]: orc.OrcInputFormat (OrcInputFormat.java:setSearchArgument(397)) - ORC pushdown predicate: leaf-0 = (IN id 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81), leaf-1 = (EQUALS tenantname CERTTEST41), expr = (and leaf-0 leaf-1) 2021-06-27 00:15:11,165 INFO [HiveServer2-Handler-Pool: Thread-17672971]: orc.ReaderImpl (ReaderImpl.java:rowsOptions(478)) - Reading ORC rows from s3a://cf-s3-3b4640e1-bc89-4021-b54d-3400fee16fad/fz/patient/staging/day=2021_6_26/000000_0_copy_391 with {include: [true, true, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, true, false, false, true, false, false, true, true], offset: 0, length: 5054, sarg: leaf-0 = (IN id 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81 66d09a49-7328-4ef0-979d-c9c9f9d0a485 9e437fb1-fae4-4e54-9e03-ef261ad5f04a a5a93187-c82e-429e-a0af-2f188fce9e81), leaf-1 = (EQUALS tenantname CERTTEST41), expr = (and leaf-0 leaf-1), columns: ['null', 'id', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'null', 'ts', 'null', 'null', 'clinicalunit', 'null', 'null', 'collectorid', 'tenantname']} 2021-06-27 00:15:11,172 INFO [HiveServer2-Background-Pool: Thread-27798033]: client.TezClient (TezClient.java:submitDAGSession(501)) - Submitted dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64467, dagName=FROM feed.PhysioData_FEED_4fe...sourceDevice(Stage-1) 2021-06-27 00:25:27,642 INFO [HiveServer2-Background-Pool: Thread-27824113]: client.TezClient (TezClient.java:submitDAGSession(429)) - Submitting dag to TezSession, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64467, dagName=FROM feed.PhysioData_FEED_a45...sourceDevice(Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=hive_20210627002525_2a3fc757-941e-438f-9221-b3d9c0eb3e39 } 2021-06-27 00:25:27,642 INFO [HiveServer2-Background-Pool: Thread-27823820]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=TezRunVertex.Reducer 2 start=1624753524808 end=1624753527642 duration=2834 from=org.apache.hadoop.hive.ql.exec.tez.monitoring.TezJobMonitor> 2021-06-27 00:25:27,645 INFO [HiveServer2-Background-Pool: Thread-27823820]: monitoring.TezJobMonitor$UpdateFunction (TezJobMonitor.java:update(137)) - Map 1: 1/1 Map 3: 1/1 Reducer 2: 1/1 2021-06-27 00:25:27,645 INFO [HiveServer2-Background-Pool: Thread-27823820]: log.PerfLogger (PerfLogger.java:PerfLogEnd(177)) - </PERFLOG method=TezRunDag start=1624753520200 end=1624753527645 duration=7445 from=org.apache.hadoop.hive.ql.exec.tez.monitoring.TezJobMonitor> 2021-06-27 00:25:27,648 INFO [HiveServer2-Background-Pool: Thread-27824113]: client.TezClientUtils (TezClientUtils.java:getAMProxy(850)) - Application not running, applicationId=application_1622134593468_64467, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64467/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 2021-06-27 00:25:27,648 INFO [HiveServer2-Background-Pool: Thread-27824113]: exec.Task (SessionState.java:printInfo(1106)) - Tez session was closed. Reopening... 2021-06-27 00:25:27,648 INFO [HiveServer2-Background-Pool: Thread-27824113]: tez.TezSessionPoolManager (TezSessionPoolManager.java:close(226)) - Closing tez session default? false 2021-06-27 00:25:27,648 INFO [HiveServer2-Background-Pool: Thread-27824113]: tez.TezSessionState (TezSessionState.java:close(294)) - Closing Tez Session 2021-06-27 00:25:27,648 INFO [HiveServer2-Background-Pool: Thread-27824113]: client.TezClient (TezClient.java:stop(518)) - Shutting down Tez Session, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64467 2021-06-27 00:25:27,649 INFO [HiveServer2-Background-Pool: Thread-27824113]: client.TezClientUtils (TezClientUtils.java:getAMProxy(850)) - Application not running, applicationId=application_1622134593468_64467, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64467/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 2021-06-27 00:25:27,649 INFO [HiveServer2-Background-Pool: Thread-27824113]: client.TezClient (TezClient.java:stop(532)) - Failed to shutdown Tez Session via proxy org.apache.tez.dag.api.SessionNotRunning: Application not running, applicationId=application_1622134593468_64467, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://**:8088/proxy/application_1622134593468_64467/, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 at org.apache.tez.client.TezClientUtils.getAMProxy(TezClientUtils.java:851) at org.apache.tez.client.TezClient.getAMProxy(TezClient.java:736) at org.apache.tez.client.TezClient.stop(TezClient.java:524) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.close(TezSessionState.java:296) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.close(TezSessionPoolManager.java:228) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeAndOpen(TezSessionPoolManager.java:353) at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:503) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:196) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:177) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:96) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1777) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1511) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1308) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1175) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76) at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:273) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2021-06-27 00:25:27,649 INFO [HiveServer2-Background-Pool: Thread-27824113]: client.TezClient (TezClient.java:stop(537)) - Could not connect to AM, killing session via YARN, sessionName=HIVE-d4bec027-6c39-4549-86dd-305da87a612d, applicationId=application_1622134593468_64467 ... View more