Kafka Upgrade Consumer/Producer compatibility

[#1]

Hello,

My team will be upgrading our cluster from 5.x to 6.x soon and we have some doubts regarding the consumer and producer compatibility after the cluster has been upgraded.

This upgrade will lead to a change on Kafka and Flume versions as shown below:

Kafka	1.0.1	2.2.1
Flume	1.6.0	1.9.0

It was not very clear to me which should be the steps to perform on the consumers and producers to avoid them not being able to communicate with Kafka after the upgrade.

From the information I have gathered, it seems that the producers are forward compatible, meaning that initially it would not be required for them to perform any change, contrariwise the consumers must be updated for them to get information from Kafka after the upgrade.

To summarize:

• No changes required on the producer side;
• Code update required on the consumer side.

Are the assumptions above correct?

Thanks in advance!

[#2]

Just a note for future that Flume is deprected and Cloudera Flow Management with Apache NiFi is the path forward for such workloads.

https://docs.cloudera.com/runtime/7.0.3/release-notes/topics/cdpdc-rt-updated-cdh-components.html

Versions are here:

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_63_packaging.html