- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
ConvertCSVToAvro with InferAvroScheme
- Labels:
-
Apache NiFi
Created ‎06-19-2018 10:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
-- New day - new problem.
I download the CSV file into the InferAvroScheme to determine the scheme of the avro file.
After that file's sent to the ConvertCSVToAvro where it is converted.
1. InferAvroSchema incorrectly defines the scheme, outputting in a single cell all the values.
2. ConvertCSVToAvro "Cannot find schema", although I indicated ${inferred.avro.schema} in Record schema.
Created on ‎06-19-2018 02:20 PM - edited ‎08-17-2019 06:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think you are missing small configuration in InferAvroSchema processor
Change Schema Output Destination
flowfile-attribute
Instead of flowfile-content then the inferred.avro.schema attribute will be added to the flowfile.
Writes Attributes:
Name | Description |
---|---|
inferred.avro.schema | If configured by "Schema output destination" to write to an attribute this will hold the resulting Avro schema from inferring the incoming FlowFile content. |
Once you make this changes then ConverCSVto Avro processor will be able to find out inferred.avro.schema attribute.
Created on ‎06-19-2018 02:20 PM - edited ‎08-17-2019 06:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think you are missing small configuration in InferAvroSchema processor
Change Schema Output Destination
flowfile-attribute
Instead of flowfile-content then the inferred.avro.schema attribute will be added to the flowfile.
Writes Attributes:
Name | Description |
---|---|
inferred.avro.schema | If configured by "Schema output destination" to write to an attribute this will hold the resulting Avro schema from inferring the incoming FlowFile content. |
Once you make this changes then ConverCSVto Avro processor will be able to find out inferred.avro.schema attribute.
Created ‎06-19-2018 03:56 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, thanks!
