Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

migrating from open source kafka to cloudera kafka

Highlighted

migrating from open source kafka to cloudera kafka

New Contributor

we are trying to do an in place upgrade of our Open source kafka to cloudera kafka. the installation and configuration steps seem to work fine. we point to the same data dir /app/respository/kafka and /app/respository/zookeeper as we did when we had the open source. Once migrated i am able to see the same topics when i do a list on the cloudera kafka instance. we are facing the below problem though.

 

Before migration i pusblished a few messages with a rentention of 5 days. after migration, i tried to so a console consumer from begenning on the same topic but i am not able to get the messages that i had published prior to the migration.

 

how to get old data from the new cloudera kafka? Had opened a case with cloudera which is still open, they have suggested using mirror maker but i read that when using mirror maker offsets are not maintained. i would think that would affect the consumers if they want to go back and consume old data.

 

any thoughts/suggestions?