My Nifi ConsumeKafakRecord_0_11 throws the following error
Failed to parse message from Kafka using the configured Record Reader. Will route message as its own FlowFile to the 'parse.failure' relationship: org.apache.nifi.schema.access.SchemaNotFoundException: Could not find a schema with identifier 488
EVN: Nifi 1.4 with kafka connect and confluent schema registry service and the backend database to monitor is mysql
When: This error occurred only recently when we changed to use nifi EL to specify kafka broker urls and topics in our ConsumeKafakRecord_0_11. The kafka topic is created through kafka connect with schema registry enabled.
The issue here is the topic that throws this error has 3 versions in schema registry and that identifier 488 is in version 2 while ConsumeKafakRecord_0_11 issued to call to schema registry with "latest" that is latest version 3 which has a higher identifier 529.
The question is how to fix this issue? I am trying to understand how ConsumeKafakRecord_0_11 determines to use 488(version 2) instead of 529(version 3).
This seems to be a nifi 1.4/1.5 issue. This issue is reproducible in both nifi 1.4 and 1.5. However, it works well with nifi 1.6 arvo recordreader.