Support Questions
Find answers, ask questions, and share your expertise

Why is Convert Record leaving two fields null, and ignoring attributes

My simple flow:

108160-1556128273261.png


The input XML:

108175-1556128310926.png


The output JSON:

108176-1556128353974.png


ConvertRecord Configuration:

108135-1556128385095.png


XMLReader Configuration:

108177-1556128419779.png


Several questions arise:

  1. Why are position and ncsmTrackData null?
  2. Why are all the attributes on fltdMessage being ignored?
  3. How can I see the inferred schema?
  4. What is a proper schema to correctly transform those two null fields, (along with the rest of the transformation as it currently looks ok)?



6 REPLIES 6

I've added a simple template that someone can use to replicate this. Test_Convert_Record_2.xml

Explorer

@David Sargrad , I faced the same issue and I believe that the nested structure of your XML is causing inferredSchema to fill null in those attributes. I came across this post, it suggested using a XSLT file to transform the XML directly to JSON. The inferred Schema works great for flat XML files.

You might want to give it a try or you can manually define a schema for the file you want to convert.

Here is how you can use the XSLT to transform the XML.

108263-untitled.png

Configuration of TransformXML processor

Save the XSLT file and add the path to the file at XSLT file name

108283-untitled.png

Hi @Sumit Das


Thanks for your response. I came to the same conclusion. I have indeed used XSLT before along with TransformXML. I was hoping that the new capability with XMLReader would give me an alternative option.


Given the current state of the inference capability, I think that you are right.


Thanks again for taking the time to respond.

I did write an issue on the NIFI issue board: https://issues.apache.org/jira/browse/NIFI-6241. Hopefully this will get some attention.

Explorer

@David Sargrad , Would you mind sharing the source of your XML? I want to see if I missed something. Also, I could not find a way to have a look at the schema. Write Schema to attribute is disabled on inferSchema. Even InferavroSchema processor supports CSV and JSON only at the moment. Please share if you find a way to look at the inferred schema in future.

If it is a bug, lets hope it gets fixed soon.

Please see the template i added in my comment above.. it has the source generator that you need (Test_Convert_Record_2.xml)

108284-1556204459766.png

Explorer

I fiddled with the XML and I found that the issue is having data in form of XML attributes. For example if the longitude and latitude data is in this format, you get the desired output.

108214-xml.png

Seems like there is some issue in the XMLReader. I upvoted your JIRA issue. Lets wait and see. Thanks for your response.