Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Why is Convert Record leaving two fields null, and ignoring attributes

Highlighted

Why is Convert Record leaving two fields null, and ignoring attributes

New Contributor

My simple flow:

108160-1556128273261.png


The input XML:

108175-1556128310926.png


The output JSON:

108176-1556128353974.png


ConvertRecord Configuration:

108135-1556128385095.png


XMLReader Configuration:

108177-1556128419779.png


Several questions arise:

  1. Why are position and ncsmTrackData null?
  2. Why are all the attributes on fltdMessage being ignored?
  3. How can I see the inferred schema?
  4. What is a proper schema to correctly transform those two null fields, (along with the rest of the transformation as it currently looks ok)?



6 REPLIES 6

Re: Why is Convert Record leaving two fields null, and ignoring attributes

New Contributor

I've added a simple template that someone can use to replicate this. Test_Convert_Record_2.xml

Re: Why is Convert Record leaving two fields null, and ignoring attributes

New Contributor

@David Sargrad , I faced the same issue and I believe that the nested structure of your XML is causing inferredSchema to fill null in those attributes. I came across this post, it suggested using a XSLT file to transform the XML directly to JSON. The inferred Schema works great for flat XML files.

You might want to give it a try or you can manually define a schema for the file you want to convert.

Here is how you can use the XSLT to transform the XML.

108263-untitled.png

Configuration of TransformXML processor

Save the XSLT file and add the path to the file at XSLT file name

108283-untitled.png

Re: Why is Convert Record leaving two fields null, and ignoring attributes

New Contributor

Hi @Sumit Das


Thanks for your response. I came to the same conclusion. I have indeed used XSLT before along with TransformXML. I was hoping that the new capability with XMLReader would give me an alternative option.


Given the current state of the inference capability, I think that you are right.


Thanks again for taking the time to respond.

I did write an issue on the NIFI issue board: https://issues.apache.org/jira/browse/NIFI-6241. Hopefully this will get some attention.

Re: Why is Convert Record leaving two fields null, and ignoring attributes

New Contributor

@David Sargrad , Would you mind sharing the source of your XML? I want to see if I missed something. Also, I could not find a way to have a look at the schema. Write Schema to attribute is disabled on inferSchema. Even InferavroSchema processor supports CSV and JSON only at the moment. Please share if you find a way to look at the inferred schema in future.

If it is a bug, lets hope it gets fixed soon.

Re: Why is Convert Record leaving two fields null, and ignoring attributes

New Contributor

Please see the template i added in my comment above.. it has the source generator that you need (Test_Convert_Record_2.xml)

108284-1556204459766.png

Re: Why is Convert Record leaving two fields null, and ignoring attributes

New Contributor

I fiddled with the XML and I found that the issue is having data in form of XML attributes. For example if the longitude and latitude data is in this format, you get the desired output.

108214-xml.png

Seems like there is some issue in the XMLReader. I upvoted your JIRA issue. Lets wait and see. Thanks for your response.