Can't consume avro messages, whcih are published by PublishKafkaRecord_0_10

[#1]

Hello,

I work with NiFi v1.3 and publish avro messages by using PublishKafkaRecord_0_10 to Kafka.

However, when I use kafka-avro-console-consumer to consume avro from Kafka topic, I saw below errors:

ERROR Unknown error when running consumer: (kafka.tools.ConsoleConsumer$:105) org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1 Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

When I use kafka-console-consumer to consume, everything goes well.

Any ideas to make the avro messages published from NiFi deserializable for Confluent Avro deserializer?

Thanks.

[#2]

This was answered in another question, but posting here for redundancy...

In Apache NiFi 1.3.0 this is not possible, you are publishing regular Avro with an embedded schema, and then trying to consume Avro that is expected to be in Confluent schema format.

In the master branch of Apache NiFi there is integration with Confluent schema registry, and also a new option in the "Schema Write Strategy" on the AvroRecordSetWriter - "Confluent Schema Registry Reference".

When using PublishKafkaRecord_0_10 with an AvroRecordSetWriter that is configured with "Confluent Schema Registry Reference" you should be publishing Avro that can then be deserialized using the Confluent deserializer.