NiFi: LookupRecord 1.3.0 Flow Example

[#1]

Hi Team,

We have few use-cases that require using CSV tables as a lookups. Based on docs, it is something LookupRecord processor is designed to do. Do we have any example flows / templates / tutorials that we can use?

For a bit more context, here's what we're trying to achieve:

• We have a flowfile with certain attribute name
• Value of this attribute represents a key that we'd like to search for
• We have a CSV file with 3 columns and a header (let's say key, col1, col2)
• We'd like to add col1 and col2 as attributes and their respective values as attribute values for a given key

[#2 · Accepted solution]

I think you might be able to do this with LookupAttribute and SimpleCsvFileLookupService, but you might need to do two separate look ups...

With the SimpleCsvFileLookupService you provide it with a path to a CSV file and then a key column and a value column, and it then loads those two columns into a lookup map. So you would probably need two instances of this service, one where the lookup column was "key" and value column was "col1" and then the second instance where the look up column was "key" and the value column was "col2".

With that setup you could then have two LookupAttribute processors in a row where each one used one of the loop up services above.

LookupRecord would be used if you have record-oriented data in your flow file (csv, json, avro) and you want to enrich each record based on a lookup. I'm assuming you want flow file attributes based on your description, which would be LookupAttribute.

[#3]

Hi @Bryan Bende How to get more than one column value from lookup file? PFB details.

InputFile

ID, Name
1,Elon
2,Mark
3,Jeff   

Lookup File

ID,Company,Domain,HQ
1,Tesla,Automobile,Palo Alto
2,Facebook,SocialMedia,Menlo Park
3,Amazon,ECommerce,Seattle

Output

ID,Name,Company,Domain,HQ
1,Elon,Tesla,Automobile,Palo Alto
2,Mark,Facebook,SocialMedia,Menlo Park
3,Jeff,Amazon,ECommerce,Seattle

Result RecordPath allows only one value. Any tips to get this kind of output?