Skip to main content
Eldho
Eldho
Bronze III
March 3, 2020
Solved

Reference table lookup using NiFi

We are loading data from oracle table to Hive using NiFi. At the time of loading we will need to do some reference table lookup.
(e.g Customer data has Country_code = IN in Oracle, while loading Customer data in HDFS,
lookup using Country_code=In will be looked up in reference table and new column with value Country_detail = INDIA will be populated in Customer data).

 

Could you please guide me how it can be done?

    TimothySpann
    TimothySpannAnswer
    Platinum ⭐️⭐️⭐️
    March 3, 2020

    Use LookupRecord processor

     

    it can use JDBC sources

     

    http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.3/org.apache.nifi.processors.standard.LookupRecord/

    View in thread

    2 replies

    TimothySpann
    Platinum ⭐️⭐️⭐️
    TimothySpannAnswer
    Platinum ⭐️⭐️⭐️
    March 3, 2020

    Use LookupRecord processor

     

    it can use JDBC sources

     

    http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.3/org.apache.nifi.processors.standard.LookupRecord/

    Translate
    TimothySpann
    Platinum ⭐️⭐️⭐️
    TimothySpann
    Platinum ⭐️⭐️⭐️
    March 3, 2020

    https://community.cloudera.com/t5/Community-Articles/ETL-With-Lookups-with-Apache-HBase-and-Apache-NiFi/ta-p/248243

     

    That is an HBase example, just use a different one.

     

    We also teach how to do the lookup / update record processors in our CDF Workshop roadshow

    Translate
    Terms & ConditionsAccessibility statement