Created 12-14-2016 08:31 AM
Hello there, somehow I dont see AvroRecordConverter in NiFi's available Processors. I need to be able to convert an incoming source file (CSV/XML/JSON) to a generic schema. However, the problem is I am not sure how to - 1. Convert a AVRO 'record' type to another and 2. When using ConvertAvroSchema how do we specify the dynamic properties as specified in the NiFi documentaion here - https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.kite.ConvertAvroSchema/...
Created 12-23-2016 05:41 AM
Thank you @Bryan Bende
I managed my use case with Jolt Processor, since AVRO would only transform to the schema of incoming file format, and will adjust only in terms of adding or removing some attributes. However, I was looking for a totally different outbound schema than whats incoming.
Created 12-14-2016 02:07 PM
To go from CSV or JSON to Avro you would use ConvertJsonToAvro or ConvertCsvToAvro, there is no conversion for XML currently. With ConvertJsonToAvro and ConvertCsvToAvro you can directly enter the schema in the properties of the processor, or you can attempt to use InferAvroSchema before them. InferAvroSchema will guess a schema and put it into a flow file attribute which you can then reference as ${inferred.avro.schema} in the conversion processor.
For ConvertAvroSchema you can either enter schemas directly into the properties, or you can reference flow file attributes using expression language (if you have the schemas in flow file attributes).
Created 12-23-2016 05:41 AM
Thank you @Bryan Bende
I managed my use case with Jolt Processor, since AVRO would only transform to the schema of incoming file format, and will adjust only in terms of adding or removing some attributes. However, I was looking for a totally different outbound schema than whats incoming.