question Reference table lookup using NiFi in Support Questions

Reference table lookup using NiFi

Eldho — Tue, 03 Mar 2020 16:37:52 GMT

We are loading data from oracle table to Hive using NiFi. At the time of loading we will need to do some reference table lookup.
(e.g Customer data has Country_code = IN in Oracle, while loading Customer data in HDFS,
lookup using Country_code=In will be looked up in reference table and new column with value Country_detail = INDIA will be populated in Customer data).

Could you please guide me how it can be done?

Re: Reference table lookup using NiFi

TimothySpann — Tue, 03 Mar 2020 17:18:31 GMT

Use LookupRecord processor

it can use JDBC sources

http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.3/org.apache.nifi.processors.standard.LookupRecord/

Re: Reference table lookup using NiFi

TimothySpann — Tue, 03 Mar 2020 17:19:40 GMT

https://community.cloudera.com/t5/Community-Articles/ETL-With-Lookups-with-Apache-HBase-and-Apache-NiFi/ta-p/248243

That is an HBase example, just use a different one.

We also teach how to do the lookup / update record processors in our CDF Workshop roadshow