Created 10-21-2015 03:14 PM
When a table is partitioned and bucketed and Transactions enabled on it , the number of map tasks launched by TEZ = 2 , while MR jobs still launches 72 Tasks (Table is about 17Gig). if transaction is not enabled , then the query is launching Correct number of Tez tasks, If there are any hints on why this may occur, please share.
Created 11-04-2015 03:06 PM
After compacting (Major compaction per partition) the tables (HDP 2.2.4.2-2) we got the right number of Tez mappers. So this appears to be a bug related to compaction.
Alex,
Yes, they are both submitting to the same queue.
The ACID Transactions is broken until further advisary. I am looking for more details of the reasons why it is broken. If you have details please send me a note.
Created 10-21-2015 04:55 PM
@pbalasundaram@hortonworks.com
What version of Hive and HDP are you using?
Created 11-04-2015 04:14 AM
Are the Tez jobs submitting to the same queue as MR jobs? (hive.server2.tez.default.queues, hive.server2.tez.sessions.per.default.queue)
How do Tez container settings compare with general YARN container settings?(tez.am.resource.memory.mb, tez.am.java.opts, hive.tez.container.size, hive.tez.java.opts)
Created 11-04-2015 11:45 AM
@pbalasundaram@hortonworks.com, @Ryan Templeton and @ravi@hortonworks.com found same issue. I think @ravi is interacting with engineering in order to fix or workaround it.
Created 11-04-2015 03:06 PM
After compacting (Major compaction per partition) the tables (HDP 2.2.4.2-2) we got the right number of Tez mappers. So this appears to be a bug related to compaction.
Alex,
Yes, they are both submitting to the same queue.
The ACID Transactions is broken until further advisary. I am looking for more details of the reasons why it is broken. If you have details please send me a note.
Created 07-15-2016 12:31 PM
May be related to https://issues.apache.org/jira/browse/HIVE-9977
Created 11-06-2015 04:27 PM
A few things to note:
Bottomline, we should refrain from deploying Transactions in HDP 2.2 releases.