Skip to main content
johndcal
johndcal
Gold ⭐️
June 18, 2020
Solved

NiFi: LookupRecord Could Not Parse Incoming Data

Update: template for flow titled CountryCodeLookup.xml is in GitHub: https://github.com/johndcal/NiFi-Tutorial-5-Files

Hello,

Can anyone help me with this?  Below is the data that I am trying to parse and the schema being used to parse it.  Should I be looking at something else?  Will provide more info if needed.

Thanks,

jc

 

Data:

name,loc_id,launch,norad_number,classification
NOAA 19,US,2009-02-06,33591,Weather
GOES 13,US,2006-05-24,29155,TV
SAUDISAT 1C,SA,2002-12-20,27607,Education
METEOR M2,RU,2014-07-08,40069,Weather
MEASAT 3,MY,2006-12-11,29648,TV

 

Schema:

{
"type": "record",
"namespace": "satellites",
"name": "satdata_in",
"fields": [
{ "name": "name", "type": "string" },
{ "name": "loc_id", "type": "string" },
{ "name": "launch", "type": "string" },
{ "name": "norad_number", "type": "int" },
{ "name": "classification", "type": "string" }
]
}

johndcal
johndcalAnswer
Gold ⭐️
June 19, 2020

This post does not need any more views, I have solved the problem:

 

The first line of my data is column headers.  In latter case, in CSVReader, when Schema Access Strategy is set to Use 'Schema Name' Property, then Treat First Line as Header needs to be set to True.  The property Ignore CSV Header Column Names also needs to be set to True otherwise if there are field name mismatches between the input file and the schema, the input file names will be used instead of the schema.

 

Alternative: I was able to set Schema Access Strategy to Use String Fields From Header, and that worked, but it ignores your schema, if you have one set up.

 

jc

View in thread

1 reply

johndcal
johndcalAuthorAnswer
Gold ⭐️
June 19, 2020

This post does not need any more views, I have solved the problem:

 

The first line of my data is column headers.  In latter case, in CSVReader, when Schema Access Strategy is set to Use 'Schema Name' Property, then Treat First Line as Header needs to be set to True.  The property Ignore CSV Header Column Names also needs to be set to True otherwise if there are field name mismatches between the input file and the schema, the input file names will be used instead of the schema.

 

Alternative: I was able to set Schema Access Strategy to Use String Fields From Header, and that worked, but it ignores your schema, if you have one set up.

 

jc

Translate
Terms & ConditionsAccessibility statement