New Contributor
Posts: 2
Registered: ‎01-18-2018

migrating from open source kafka to cloudera kafka

[ Edited ]

we are trying to do an in place upgrade of our Open source kafka to cloudera kafka. the installation and configuration steps seem to work fine. we point to the same data dir /app/respository/kafka and /app/respository/zookeeper as we did when we had the open source. Once migrated i am able to see the same topics when i do a list on the cloudera kafka instance. we are facing the below problem though.


Before migration i pusblished a few messages with a rentention of 5 days. after migration, i tried to so a console consumer from begenning on the same topic but i am not able to get the messages that i had published prior to the migration.


how to get old data from the new cloudera kafka? Had opened a case with cloudera which is still open, they have suggested using mirror maker but i read that when using mirror maker offsets are not maintained. i would think that would affect the consumers if they want to go back and consume old data.


any thoughts/suggestions?