Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

migrating from open source kafka to cloudera kafka

migrating from open source kafka to cloudera kafka

New Contributor

we are trying to do an in place upgrade of our Open source kafka to cloudera kafka. the installation and configuration steps seem to work fine. we point to the same data dir /app/respository/kafka and /app/respository/zookeeper as we did when we had the open source. Once migrated i am able to see the same topics when i do a list on the cloudera kafka instance. we are facing the below problem though.


Before migration i pusblished a few messages with a rentention of 5 days. after migration, i tried to so a console consumer from begenning on the same topic but i am not able to get the messages that i had published prior to the migration.


how to get old data from the new cloudera kafka? Had opened a case with cloudera which is still open, they have suggested using mirror maker but i read that when using mirror maker offsets are not maintained. i would think that would affect the consumers if they want to go back and consume old data.


any thoughts/suggestions?