I am struggling to get to grips with JOLT/AVRO to convert files into XML.
My flow is:
ConsumeKAFTA (KAFTA queue consumes JSON based files from filebeat, the flowfile created is a plain/text file.)
UpdateAttribute. (Creates an attribute mimie.type = application/json
EvaluateJsonPath (I want to route later on onm some NIFI attributes.)
Relacetext (Remove @ from any fields)
JoltTransform (Converts a number of fields to different names, eg: timestamp -> TimeCreated
InferAvroSchema (To create a AVRO schems based on the input.
ReadReader = JsonTreeReader (Uses inferred avro schema from 7 above.)
RecordWriter= XMLRecordSetWriter (Uses avroregistry, and a defined schema.)
The problem I have is that the JSON input will look very different to the XML output.
I have figured how to rename fields etc, however how to create new nested XML?
Eg: input: (Plain/text -> application/json)
“field1” : “xxxxxxxxxxxxxxxx”,
“field2” : “xxxxxxxxx”,
Eg: output needs to: (xml)
How do I define a new structure in the AVRO schema I use in the output?
As fielda, fieldb do not exist in the input, if I ‘define’ them In the AVRO schema output, they have black structures.
There seems to a number of ways of converting JSON -> XML, so:
Does the flow look generally correct?
If correct, how to I define new nested structures in the XML output using AVRO?