Member since
11-26-2019
10
Posts
0
Kudos Received
0
Solutions
11-27-2019
11:42 AM
Now all is fine. After deleting the folders and restarting the service the issue is resolved. Thanks very much for the support
... View more
11-27-2019
09:23 AM
I implemented the same thing in Production and reassignment is showing as completed. But old segmented logs and future folder is not getting deleted. From the logs i can see as below INFO [ReplicaAlterLogDirsThread-1]: Partition XXXX-1 has an older epoch (102) than the current leader. Will await the new LeaderAndIsr state before resuming fetching. (kafka.server.ReplicaAlterLogDirsThread) INFO [Partition XXX-1 broker=3] XXX-1 starts at Leader Epoch 106 from offset 0. Previous Leader Epoch was: 105 (kafka.cluster.Partition) After restarting the service, it tried to copy the data from one folder to other folder for some time and then stopped
... View more
11-27-2019
08:46 AM
With ANY it worked. And also i need to restart the kafka service to delete the old segment logs. Thanks very much for your support
... View more
11-27-2019
08:16 AM
Hi, i was able to replicate the same issue in Test environment. I did rerun the --execute command for the same json file but still the same.Nothing changed after re-executing the command once again. Any other suggestions?
... View more
11-26-2019
06:54 PM
I ran the reassignment previously using the below command with the json which i shared earlier. Do you want to run the same once again? /kafka-reassign-partitions.sh --execute --bootstrap-server XXXX:9092 --reassignment-json-file ./test.json --zookeeper XXXXX:2181 As this is production i want to confirm once again
... View more
11-26-2019
06:35 PM
Before Reassignment the replicas are as below for Topic XXX Topic: XXXX Partition: 1 Leader: 3 Replicas: 3,0,1 Original Reassignment json file: {"version":1,"partitions":[{"topic":"XXXXX","partition":1,"replicas":[3,0,8],"log_dirs":["/data1/kafka","/data3/kafka","/data1/kafka"]}]} For 3 node am moving the partition 1 data from data2 to data1 and for 0 node am moving the partition 1 data from data1 to data3 and for node 8 it is new node Status of Verify as below Status of partition reassignment: Reassignment of partition XXXX-1 completed successfully Reassignment of replica XXXX-1-3 is still in progress Reassignment of replica XXXX-1-0 is still in progress Reassignment of replica XXXX-1-8 completed successfully I didnt set any values which you mentioned. The problem now here is old logs are not getting deleted and in the logs as seeing error as below INFO [Partition XXXX-1 broker=3] XXXX-1 starts at Leader Epoch 101 from offset 10647332982. Previous Leader Epoch was: 100 (kafka.cluster.Partition) INFO [Partition XXXX-tapsampleapp-1 broker=3] XXXX-tapsampleapp-1 starts at Leader Epoch 99 from offset 555229. Previous Leader Epoch was: 98 (kafka.cluster.Partition) INFO [ReplicaAlterLogDirsThread-1]: Partition XXXX-1 has an older epoch (100) than the current leader. Will await the new LeaderAndIsr state before resuming fetching. (kafka.server.ReplicaAlterLogDirsThread) Intially it created one future folder and tried to do reasignment and then stopped suddenly. Now my main concern is old logs are not getting deleted and file system is getting full
... View more
11-26-2019
06:07 PM
Can you please let me know whether the issue resolved?Am also facing the same issue.Reassignment is still running since 2 days. I ran the reassignment without throttling. Also after this am seeing old logs files still exists and retention.ms and retention.bytes is not working
... View more