NiFi: LookupRecord Could Not Parse Incoming Data

johndcal — Thu, 18 Jun 2020 15:30:45 GMT

Update: template for flow titled CountryCodeLookup.xml is in GitHub: https://github.com/johndcal/NiFi-Tutorial-5-Files

Hello,

Can anyone help me with this? Below is the data that I am trying to parse and the schema being used to parse it. Should I be looking at something else? Will provide more info if needed.

Thanks,

Data:

name,loc_id,launch,norad_number,classification
NOAA 19,US,2009-02-06,33591,Weather
GOES 13,US,2006-05-24,29155,TV
SAUDISAT 1C,SA,2002-12-20,27607,Education
METEOR M2,RU,2014-07-08,40069,Weather
MEASAT 3,MY,2006-12-11,29648,TV

Schema:

{
"type": "record",
"namespace": "satellites",
"name": "satdata_in",
"fields": [
{ "name": "name", "type": "string" },
{ "name": "loc_id", "type": "string" },
{ "name": "launch", "type": "string" },
{ "name": "norad_number", "type": "int" },
{ "name": "classification", "type": "string" }
]
}

Re: NiFi: LookupRecord Could Not Parse Incoming Data

johndcal — Fri, 19 Jun 2020 14:37:26 GMT

This post does not need any more views, I have solved the problem:

The first line of my data is column headers. In latter case, in CSVReader, when Schema Access Strategy is set to Use 'Schema Name' Property, then Treat First Line as Header needs to be set to True. The property Ignore CSV Header Column Names also needs to be set to True otherwise if there are field name mismatches between the input file and the schema, the input file names will be used instead of the schema.

Alternative: I was able to set Schema Access Strategy to Use String Fields From Header, and that worked, but it ignores your schema, if you have one set up.

question Re: NiFi: LookupRecord Could Not Parse Incoming Data in Support Questions

NiFi: LookupRecord Could Not Parse Incoming Data

Re: NiFi: LookupRecord Could Not Parse Incoming Data