Support Questions

Find answers, ask questions, and share your expertise

NiFi: LookupRecord Could Not Parse Incoming Data

avatar
Explorer

Update: template for flow titled CountryCodeLookup.xml is in GitHub: https://github.com/johndcal/NiFi-Tutorial-5-Files

Hello,

Can anyone help me with this?  Below is the data that I am trying to parse and the schema being used to parse it.  Should I be looking at something else?  Will provide more info if needed.

Thanks,

jc

 

Data:

name,loc_id,launch,norad_number,classification
NOAA 19,US,2009-02-06,33591,Weather
GOES 13,US,2006-05-24,29155,TV
SAUDISAT 1C,SA,2002-12-20,27607,Education
METEOR M2,RU,2014-07-08,40069,Weather
MEASAT 3,MY,2006-12-11,29648,TV

 

Schema:

{
"type": "record",
"namespace": "satellites",
"name": "satdata_in",
"fields": [
{ "name": "name", "type": "string" },
{ "name": "loc_id", "type": "string" },
{ "name": "launch", "type": "string" },
{ "name": "norad_number", "type": "int" },
{ "name": "classification", "type": "string" }
]
}

1 ACCEPTED SOLUTION

avatar
Explorer

This post does not need any more views, I have solved the problem:

 

The first line of my data is column headers.  In latter case, in CSVReader, when Schema Access Strategy is set to Use 'Schema Name' Property, then Treat First Line as Header needs to be set to True.  The property Ignore CSV Header Column Names also needs to be set to True otherwise if there are field name mismatches between the input file and the schema, the input file names will be used instead of the schema.

 

Alternative: I was able to set Schema Access Strategy to Use String Fields From Header, and that worked, but it ignores your schema, if you have one set up.

 

jc

View solution in original post

1 REPLY 1

avatar
Explorer

This post does not need any more views, I have solved the problem:

 

The first line of my data is column headers.  In latter case, in CSVReader, when Schema Access Strategy is set to Use 'Schema Name' Property, then Treat First Line as Header needs to be set to True.  The property Ignore CSV Header Column Names also needs to be set to True otherwise if there are field name mismatches between the input file and the schema, the input file names will be used instead of the schema.

 

Alternative: I was able to set Schema Access Strategy to Use String Fields From Header, and that worked, but it ignores your schema, if you have one set up.

 

jc