Unable to get correct confluent schema registry schema id for kafka key

hegdemahendra — Wed, 04 Sep 2024 12:34:18 GMT

Hello Experts,

I have a nifi kafka consumer (ConsumeKafka_2_6) where the kafka message body (value/flow file content) and message key (kafka.key in flow file attributes) both are avro serialized as per the Confluent kafka way of serializing.

When we use "ConvertRecord + AvroReader CS + ConfluentSchemaRegistry CS" to convert message body (value/flow file content) , it works fine as it is deserilising the magic byte and schema id to the correct value.

But we are trying below to deserilise kafka.key as well the same as value.

Bringing the FF attribute kafka key to content (using ReplaceText processor with ${kafka.key}) and use "onvert Record + Avro Reader + ConfluentSchemaRegistry" to deserialize, in this case nifi is resulting in a wrong schema id - 3567 instead of the correct schema id 3545.

Is it happening because when Nifi reads kafka.key (originally byte array) and pushes to downstream in FF as FF attribute and which is string?

Is there any other way I can fix this or any other right approach?

Thanks in advance!

@bbende @MattWho @mattw

Mahendra

Re: Unable to get correct confluent schema registry schema id for kafka key

hegdemahendra — Thu, 05 Sep 2024 05:18:03 GMT

@araujo @bbende @MattWho - do you have any suggestions?

question Unable to get correct confluent schema registry schema id for kafka key in Support Questions

Unable to get correct confluent schema registry schema id for kafka key

Re: Unable to get correct confluent schema registry schema id for kafka key