Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

ConvertCSVToAvro with InferAvroScheme

Explorer
-- New day - new problem.

I download the CSV file into the InferAvroScheme to determine the scheme of the avro file.
After that file's sent to the ConvertCSVToAvro where it is converted.
1. InferAvroSchema incorrectly defines the scheme, outputting in a single cell all the values.
2. ConvertCSVToAvro "Cannot find schema", although I indicated ${inferred.avro.schema} in Record schema.


114.jpg112.jpg113.jpg11.jpg
1 ACCEPTED SOLUTION

Super Guru
@Vladislav Shcherbakov

I think you are missing small configuration in InferAvroSchema processor

Change Schema Output Destination

flowfile-attribute

Instead of flowfile-content then the inferred.avro.schema attribute will be added to the flowfile.

Writes Attributes:

NameDescription
inferred.avro.schemaIf configured by "Schema output destination" to write to an attribute this will hold the resulting Avro schema from inferring the incoming FlowFile content.

77751-inferavroschema.png

Once you make this changes then ConverCSVto Avro processor will be able to find out inferred.avro.schema attribute.

View solution in original post

2 REPLIES 2

Super Guru
@Vladislav Shcherbakov

I think you are missing small configuration in InferAvroSchema processor

Change Schema Output Destination

flowfile-attribute

Instead of flowfile-content then the inferred.avro.schema attribute will be added to the flowfile.

Writes Attributes:

NameDescription
inferred.avro.schemaIf configured by "Schema output destination" to write to an attribute this will hold the resulting Avro schema from inferring the incoming FlowFile content.

77751-inferavroschema.png

Once you make this changes then ConverCSVto Avro processor will be able to find out inferred.avro.schema attribute.

Explorer

Yes, thanks!