Created on 05-17-2018 02:31 AM - edited 09-16-2022 06:13 AM
Hello,
Recently i have made UAT cluster of KAFKA with three brokers. the problem is min.insync.replicas is not taking any effect. following things are happeninig :
1. in server.properties there is no mention of min.insync.replicas, whenever i am creating any topic and whatever replication number i am providing ,in synch replica is just copying that. so if i create topic test with replica 3 in synch replica is also 3. even if i am creating replication with 2 then in synch is also 2.
Partition Detail
Partition First Offset Last Offset Size Leader Replicas In Sync Replicas Preferred Leader? Under Replicated?
0 | 0 | 0 | 0 | 2 | 2,3,1 | 1,2,3 | Yes | No |
1 | 0 | 0 | 0 | 3 | 3,1,2 | 1,2,3 | Yes | No |
2 | 0 | 0 | 0 | 1 | 1,2,3 | 1,2,3 | Yes | No |
2. i tried to put min.insync.replicas=2 in server.properties of all the brokers and tried creating the topic with replication 3, here i was expecting in synch replica to be 2 but no its again 3.
3. then i tried to create topic by providing config with this :
kafka-topics.sh --create --zookeeper machine1:2181,machine2:2181,machine3:2181 --replication-factor 3 --partitions 3 --config min.insync.replicas=1 --topic syncdemo
here i thought this time it will show in synch replica as 1 but no ,it stills 3.
4. lastly i tried to alter it to provide 2 instead of 1 to check whether alter make it work by below :
kafka-topics.sh --alter --zookeeper machine1:2181,machine2:2181,machine3:2181 --topic syncdemo --config min.insync.replicas=1 ,
but no effect.
its really very frustrating that none of the ways are making it work. please provide the solution,thanks in advance.
i am using : CDH 5.7 and apache Kafka : kafka_2.11-1.0.0 , zookeeper-3.4.5-cdh5.7.0 on redhat 6.5.
Created 05-17-2018 10:15 AM
The output you see is showing that all three replicas are in sync. The min.insync.replicas governs whether you are able to produce or consume if that number in the ISR is less than min.insync.replicas. Have you tried to shut down one broker and see if you can still produce or consume to the topic? How are you validating when min.insync.replicas threshold is crossed?
-pd
Created 05-17-2018 10:15 AM
The output you see is showing that all three replicas are in sync. The min.insync.replicas governs whether you are able to produce or consume if that number in the ISR is less than min.insync.replicas. Have you tried to shut down one broker and see if you can still produce or consume to the topic? How are you validating when min.insync.replicas threshold is crossed?
-pd
Created 05-17-2018 11:12 PM
you are correct, i tested. i thought it shows min.insynch.replicas also,thanks for help.