Support Questions

Find answers, ask questions, and share your expertise

Unable to get correct confluent schema registry schema id for kafka key

avatar
Master Collaborator

Hello Experts,

I have a nifi kafka consumer (ConsumeKafka_2_6) where the kafka message body (value/flow file content) and message key (kafka.key in flow file attributes) both are avro serialized as per the Confluent kafka way of  serializing.

When we use "ConvertRecord + AvroReader CS + ConfluentSchemaRegistry CS"   to convert message body (value/flow file content) , it works fine as it is deserilising the magic byte and schema id to the correct value.

 

But we are trying below to deserilise kafka.key as well the same as value.

Bringing the FF attribute kafka key to content (using ReplaceText processor with ${kafka.key}) and use "onvert Record + Avro Reader + ConfluentSchemaRegistry" to deserialize, in this case nifi is resulting in a wrong schema id - 3567 instead of the correct schema id 3545.

Is it happening because when Nifi reads kafka.key (originally byte array) and pushes to downstream in FF as FF attribute and which is string?

Is there any other way I can fix this or any other right approach?

 

Thanks in advance!

@bbende @MattWho @mattw 

Mahendra

1 REPLY 1

avatar
Master Collaborator

@araujo @bbende @MattWho - do you have any suggestions?