HIVE insert/update/delete

avijeetd — Fri, 12 May 2017 12:51:28 GMT

Hi All,

I tried HIVE insert/update/delete operations, I see that it launches a tez job and bit slow. I have a couple of questions -

1. when we use hive streaming processors (in nifi, streamsets) does it insert record by record? I believe no. just to confirm.

2. it seems bit risky to design an acid hive table, is there an approach where it can be used (such as IOT ) safely without having locking/concurrency issues etc.

Thanks,

Avijeet

Re: HIVE insert/update/delete

bhagan — Fri, 09 Jun 2017 00:28:30 GMT

In Nifi, you can use the PutHiveStreaming processor, and it is designed to commit transactions in batch, which is configurable.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hive-nar/1.2.0/org.apache.nifi.processors.hive.PutHiveStreaming/index.html

I think risky is not the correct term for using an ACID table, but careful may be better; that is, with careful design and configuration, you can avoid locking issues. Be sure to review the Hive documentation on this:

https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions

question Re: HIVE insert/update/delete in Archives of Support Questions (Read Only)

HIVE insert/update/delete

Re: HIVE insert/update/delete