Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

LookUpRecord and SimpleCsvFileLookupService in NiFi

Highlighted

LookUpRecord and SimpleCsvFileLookupService in NiFi

Hi There,

I am trying to lookup records from a CSV file using LookupRecord and SimpleCsvFileLookupService but I am unable to get the final record with both values. Please find below details.

Input File

ID, Name
1,Elon
2,Mark
3,Jeff    

Lookup File

ID,Company
1,Tesla
2,Facebook
3,Amazon 

Expected Output

ID,Name,Company
1,Elon,Tesla
2,Mark,Facebook
3,Jeff,Amazon

AvroSchema

{
  "name": "ceodata",
  "namespace": "nifi",
  "type": "record",
  "fields": [
    { "name": "ID", "type": "string" },
    { "name": "NAME", "type": "string" },
    { "name": "COMPANY", "type": "string" }
  ]
}

LookupRecord Processor Config

CSVReader Config

CSVWriter Config

LookupFromCSVFileService

Please let me know what's wrong in these configurations.

@jfrazee @Bryan Bende @Abdelkrim Hadjidj @Timothy Spann @Aldrin Piri

3 REPLIES 3
Highlighted

Re: LookUpRecord and SimpleCsvFileLookupService in NiFi

Hi @spdvnz

Your input and output CSV schema for the LookupRecord should be different. In the output schema you should add a field 'Company' that the processor will populate.

Take a look at this example where I added the field city in the output schema : https://medium.com/@abdelkrim.hadjidj/data-flow-enrichment-with-apache-nifi-d221f1dde419

Highlighted

Re: LookUpRecord and SimpleCsvFileLookupService in NiFi

Super Guru

If that is your output schema, then the Result RecordPath in LookupRecord should be /COMPANY not /Company.

Re: LookUpRecord and SimpleCsvFileLookupService in NiFi

Thanks for replying @Abdelkrim Hadjidj @Matt Burgess. Now I have two different schemas for input and output CSV for LookupRecord. I modified Result RecordPath to /COMPANY. When I start LookupRecord it takes in 6 flow files but doesn't return anything.

63428-lookuprecord-idle.jpg

Don't have an account?
Coming from Hortonworks? Activate your account here