Insert else update into hive table using NIFI

sateesh_reddy — Sun, 25 Jun 2017 22:40:58 GMT

I am planning to read from db2 database and do a update else insert into a hive table. what processors should be used to achive this.

Re: Insert else update into hive table using NIFI

ssahi — Mon, 26 Jun 2017 22:22:45 GMT

Hi @Sateesh Mandumula

For reading from the RDBMS, you can use processors like QueryDatabaseTable or ExecuteSQL. Hive now supports Merge which should cover your update else insert requirement, you can try using a PutHiveQL and seeing if you can issue the merge command using that. Here are some relevant links to the documentation that should help:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.3.0/org.apache.nifi.processors.standard.QueryDatabaseTable/index.html

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.3.0/org.apache.nifi.processors.standard.ExecuteSQL/index.html

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Merge

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hive-nar/1.3.0/org.apache.nifi.processors.hive.PutHiveQL/index.html

question Re: Insert else update into hive table using NIFI in Archives of Support Questions (Read Only)

Insert else update into hive table using NIFI

Re: Insert else update into hive table using NIFI