Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HIVE insert/update/delete

Solved Go to solution

HIVE insert/update/delete

Expert Contributor

Hi All,

I tried HIVE insert/update/delete operations, I see that it launches a tez job and bit slow. I have a couple of questions -

1. when we use hive streaming processors (in nifi, streamsets) does it insert record by record? I believe no. just to confirm.

2. it seems bit risky to design an acid hive table, is there an approach where it can be used (such as IOT ) safely without having locking/concurrency issues etc.

Thanks,

Avijeet

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: HIVE insert/update/delete

Expert Contributor

In Nifi, you can use the PutHiveStreaming processor, and it is designed to commit transactions in batch, which is configurable.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hive-nar/1.2.0/org.apache.nif...

I think risky is not the correct term for using an ACID table, but careful may be better; that is, with careful design and configuration, you can avoid locking issues. Be sure to review the Hive documentation on this:

https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions

1 REPLY 1
Highlighted

Re: HIVE insert/update/delete

Expert Contributor

In Nifi, you can use the PutHiveStreaming processor, and it is designed to commit transactions in batch, which is configurable.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hive-nar/1.2.0/org.apache.nif...

I think risky is not the correct term for using an ACID table, but careful may be better; that is, with careful design and configuration, you can avoid locking issues. Be sure to review the Hive documentation on this:

https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions