Member since
06-11-2018
9
Posts
1
Kudos Received
0
Solutions
08-08-2019
09:50 AM
Hi @Leonardo Araujo, don't know if you did it. About the collateral effect on 2.6.4, there is one to take in account, the Hive Acid activation change the transactional manager from : hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager To hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager The big impact is the way of Hive manage the locks even for no transactionnal tables as describe in Hive documentation about LockManager . This could dramatically increase processing times of your jobs using Hive. You could use the parameter hive.txn.strict.locking.mode to manage the behavior of the LockManager even if the documentation said that it is only available since Hive 2.2, use set -v to show presence of this parameter in Hive 1.2.1000 . You could hit 2 bugs too : HIVE-17900 : Compaction of ACID table might fail if table is partitioned by more than one column HIVE-18293 : Compaction of ACID table failed when using doas Hope this will help. Best regards, Eric.
... View more
09-18-2018
05:37 AM
Non of us is using HDP 3.0. what about HDP 2.6? If it ACID always have been "off" the last half year and 100 hive tables (text, Avro, orc) have been created. If we change to ACID as default, what other effect then performance do we need to consider? Is old table been converted to ACID tables? Will old tables still work as expected?
... View more
06-12-2018
02:26 PM
With Java you can add it to a List<Delete> and delete all at once by passing the list to the table.delete(List<>) function. If you mean you're deleting a ton of volume of data, may want to use some cron job of the jar associated with this delete. If you mean through Hbase shell, I think that only has delete a whole row or delete a single cell. I don't think it has a range-delete function unless there is a way to combine a scan and a delete.
... View more
06-11-2018
11:31 AM
Hi ducna, how are you? Regarding your mapreduce job to delete a mass rows in HBase. Can you share this job with me? I've been facing the same problem and I'm looking for some examples, what is the best approach to massively rows from HBase.
... View more
02-10-2017
05:06 PM
@Leonardo Araujo Check this link: https://wiki.apache.org/hadoop/HowManyMapsAndReduces Target one map job per block. If the file you are reading has five blocks distributed on 3 nodes (or four or five nodes) on five disks, then you should have five mappers, one for each disk.
... View more