question ConvertCSVToAvro with InferAvroScheme in Archives of Support Questions (Read Only)

ConvertCSVToAvro with InferAvroScheme

vscherbakov — Tue, 19 Jun 2018 17:22:01 GMT

-- New day - new problem.

I download the CSV file into the InferAvroScheme to determine the scheme of the avro file.
After that file's sent to the ConvertCSVToAvro where it is converted.
1. InferAvroSchema incorrectly defines the scheme, outputting in a single cell all the values.
2. ConvertCSVToAvro "Cannot find schema", although I indicated ${inferred.avro.schema} in Record schema.

Re: ConvertCSVToAvro with InferAvroScheme

Shu_ashu — Sun, 18 Aug 2019 01:46:36 GMT

@Vladislav Shcherbakov

I think you are missing small configuration in InferAvroSchema processor

Change Schema Output Destination

flowfile-attribute

Instead of flowfile-content then the inferred.avro.schema attribute will be added to the flowfile.

Writes Attributes:

Name	Description
inferred.avro.schema	If configured by "Schema output destination" to write to an attribute this will hold the resulting Avro schema from inferring the incoming FlowFile content.

Once you make this changes then ConverCSVto Avro processor will be able to find out inferred.avro.schema attribute.

Re: ConvertCSVToAvro with InferAvroScheme

vscherbakov — Tue, 19 Jun 2018 22:56:34 GMT

Yes, thanks!