Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

I want to convert the entire file to avro only if all records matches the record schema

avatar
Rising Star

I am using the 'ConvertCSVToAVRO' processor. My requirement is, I should further process the file only if all the records matches the schema. But, now, the processor generates flowfile as soon as the 'record' matches the schema. How can i continue with the flow only with 'entire file' matches the schema

1 ACCEPTED SOLUTION

avatar
Master Guru

ConvertCsvToAvro attempts to convert each record... if at least one record was converted successfully then it transfers that flow file to "success", if any records could not be converted then it also transfers a flow file to "incompatible" which has all of the original CSV in the content and a summary of the incompatible records in an attribute called "errors".

I think you need to implement a custom version of this processor that only transfers the flow file to success when the error count equals 0.

View solution in original post

1 REPLY 1

avatar
Master Guru

ConvertCsvToAvro attempts to convert each record... if at least one record was converted successfully then it transfers that flow file to "success", if any records could not be converted then it also transfers a flow file to "incompatible" which has all of the original CSV in the content and a summary of the incompatible records in an attribute called "errors".

I think you need to implement a custom version of this processor that only transfers the flow file to success when the error count equals 0.