We've recently started using CDH 6.2 and hive-2.1.1. In Our existing jobs ( in old cluster ) we've set of properties that we set by default. In that one of the property we use is `hive.exec.parallel=true`.
In the new ( hive-2.1.1/ CDH 6.2.1) cluster when we run jobs with `hive.exec.parallel=true` beeline is not writing any console logs, which shows the status of the job, application URL and other info like below
INFO : Running with YARN Application = application_1596901153465_0484 INFO : Kill Command = /opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/lib/hadoop/bin/yarn application -kill application_1596901153465_0484 INFO : Hive on Spark Session Web UI URL: http://us-east-1a-test-east-cdh-tasknode670.throtle-test.internal:45956 INFO : Query Hive on Spark job stages: [0, 1, 2] INFO : Spark job status = RUNNING INFO : Job Progress Format CurrentTime StageId_StageAttemptId: SucceededTasksCount(+RunningTasksCount-FailedTasksCount)/TotalTasksCount INFO : 2020-08-09 04:28:23,096 Stage-0_0: 0(+1)/3 Stage-1_0: 0/64 Stage-2_0: 0/64 INFO : 2020-08-09 04:28:26,106 Stage-0_0: 0(+1)/3 Stage-1_0: 0/64 Stage-2_0: 0/64 INFO : 2020-08-09 04:28:27,110 Stage-0_0: 0(+3)/3 Stage-1_0: 0/64 Stage-2_0: 0/64
I am able to get these logs **only if I disable** `hive.exec.parallel=true`
Can any one help me in getting console logs even with hive parallel execution?
Thanks @Prakashcit for pointing me to the Hive JIRA related to this bug.
I see that Fixed version is 4.0.0 does it mean I can't avail in CDH 6.2 (Hive 2.1), any idea?
Could you provide any guidance how to apply this patch to the version of Hive I am running?
Thanks in advance!
@Nagamalleswara Thanks for your response, you need to open a support case with cloudera to get the patch.
Mention about the bug and the issue, your issue will be analysed and appropriate patch will be released.