Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Regarding to activate HIVE ACID transactions on the cluster

avatar
Explorer

Hi all, I need some help here.

Regarding to activate HIVE ACID transactions on the cluster.

Nowadays, we have currently 300-node hadoop cluster - HDP, version 2.6.4.

We've been studying possible issues, collateral effects if we apply/enable this feature globally in our cluster, our legacy hive tables (about 1700) hasn't this feature and we'd like to know if have some issue/damage when enable this on the existing hive tables.

1 ACCEPTED SOLUTION

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
3 REPLIES 3

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Explorer

Thank you so much @Scott Shaw,

Best regards, Leo

avatar
Contributor

Hi @Leonardo Araujo, don't know if you did it.

About the collateral effect on 2.6.4, there is one to take in account, the Hive Acid activation change the transactional manager from :

hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager

To

hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager

The big impact is the way of Hive manage the locks even for no transactionnal tables as describe in Hive documentation about LockManager .

This could dramatically increase processing times of your jobs using Hive.

You could use the parameter hive.txn.strict.locking.mode to manage the behavior of the LockManager even if the documentation said that it is only available since Hive 2.2, use set -v to show presence of this parameter in Hive 1.2.1000 .

You could hit 2 bugs too :

HIVE-17900 : Compaction of ACID table might fail if table is partitioned by more than one column

HIVE-18293 : Compaction of ACID table failed when using doas

Hope this will help.

Best regards, Eric.