HIVE insert/update/delete

[#1]

Hi All,

I tried HIVE insert/update/delete operations, I see that it launches a tez job and bit slow. I have a couple of questions -

1. when we use hive streaming processors (in nifi, streamsets) does it insert record by record? I believe no. just to confirm.

2. it seems bit risky to design an acid hive table, is there an approach where it can be used (such as IOT ) safely without having locking/concurrency issues etc.

Thanks,

Avijeet

[#2 · Accepted solution]

In Nifi, you can use the PutHiveStreaming processor, and it is designed to commit transactions in batch, which is configurable.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hive-nar/1.2.0/org.apache.nif...

I think risky is not the correct term for using an ACID table, but careful may be better; that is, with careful design and configuration, you can avoid locking issues. Be sure to review the Hive documentation on this:

https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions