Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 12-14-2016 08:31 AM
Hello there, somehow I dont see AvroRecordConverter in NiFi's available Processors. I need to be able to convert an incoming source file (CSV/XML/JSON) to a generic schema. However, the problem is I am not sure how to - 1. Convert a AVRO 'record' type to another and 2. When using ConvertAvroSchema how do we specify the dynamic properties as specified in the NiFi documentaion here - https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.kite.ConvertAvroSchema/...
Created 12-23-2016 05:41 AM
Thank you @Bryan Bende
I managed my use case with Jolt Processor, since AVRO would only transform to the schema of incoming file format, and will adjust only in terms of adding or removing some attributes. However, I was looking for a totally different outbound schema than whats incoming.
Created 12-14-2016 02:07 PM
To go from CSV or JSON to Avro you would use ConvertJsonToAvro or ConvertCsvToAvro, there is no conversion for XML currently. With ConvertJsonToAvro and ConvertCsvToAvro you can directly enter the schema in the properties of the processor, or you can attempt to use InferAvroSchema before them. InferAvroSchema will guess a schema and put it into a flow file attribute which you can then reference as ${inferred.avro.schema} in the conversion processor.
For ConvertAvroSchema you can either enter schemas directly into the properties, or you can reference flow file attributes using expression language (if you have the schemas in flow file attributes).
Created 12-23-2016 05:41 AM
Thank you @Bryan Bende
I managed my use case with Jolt Processor, since AVRO would only transform to the schema of incoming file format, and will adjust only in terms of adding or removing some attributes. However, I was looking for a totally different outbound schema than whats incoming.