hi all I want to validate the balanced of the topics , include replica and Isr I will give some example of wrong balanced ( master01 is the zookeper server , and we defined 3 replicate ) on that example we can see that kafka brokers are missing from some Isr /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --describe --zookeeper master01:2181 --topic mno.de.pola.trump Topic:mno.de.pola.trump PartitionCount:100 ReplicationFactor:3 Configs: Topic: mno.de.pola.trump Partition: 0 Leader: 1017 Replicas: 1017,1018,1016 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 1 Leader: 1018 Replicas: 1018,1016,1017 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 2 Leader: 1016 Replicas: 1016,1017,1018 Isr: 1016,1018 Topic: mno.de.pola.trump Partition: 3 Leader: 1017 Replicas: 1017,1016,1018 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 4 Leader: 1018 Replicas: 1018,1017,1016 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 5 Leader: 1016 Replicas: 1016,1018,1017 Isr: 1016,1018,1017 Topic: mno.de.pola.trump Partition: 6 Leader: 1017 Replicas: 1017,1018,1016 Isr: 1017,1018 on the following example we can see that Leader on partitions are more on broker 1017 /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --describe --zookeeper master01:2181 --topic mno.de.pola.trump Topic:mno.de.pola.trump PartitionCount:100 ReplicationFactor:3 Configs: Topic: mno.de.pola.trump Partition: 0 Leader: 1017 Replicas: 1017,1018,1016 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 1 Leader: 1018 Replicas: 1018,1016,1017 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 2 Leader: 1017 Replicas: 1016,1017,1018 Isr: 1016,1018.1017 Topic: mno.de.pola.trump Partition: 3 Leader: 1017 Replicas: 1017,1016,1018 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 4 Leader: 1018 Replicas: 1018,1017,1016 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 5 Leader: 1016 Replicas: 1016,1018,1017 Isr: 1016,1018,1017 Topic: mno.de.pola.trump Partition: 6 Leader: 1017 Replicas: 1017,1018,1016 Isr: 1017,1018,1016 on the following example we can see that brokers are missing from Replica /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --describe --zookeeper master01:2181 --topic mno.de.pola.trump Topic:mno.de.pola.trump PartitionCount:100 ReplicationFactor:3 Configs: Topic: mno.de.pola.trump Partition: 0 Leader: 1017 Replicas: 1017,1018 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 1 Leader: 1018 Replicas: 1018,1016,1017 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 2 Leader: 1017 Replicas: 1016,1017,1018 Isr: 1016,1018,1017 Topic: mno.de.pola.trump Partition: 3 Leader: 1017 Replicas: 1017,1016 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 4 Leader: 1018 Replicas: 1018,1017,1016 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 5 Leader: 1016 Replicas: 1016,1018,1017 Isr: 1016,1018,1017 Topic: mno.de.pola.trump Partition: 6 Leader: 1017 Replicas: 1017,1018,1016 Isr: 1017,1018,1016 And so on . . . so my question - I will happy to know about some tool that can validate the output from kafka-topics.sh --describe any ideas about this example of good balanced configuration /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --describe --zookeeper master01:2181 --topic mno.de.pola.trump Topic: mno.de.pola.trump PartitionCount:100 ReplicationFactor:3 Configs: Topic: mno.de.pola.trump Partition: 0 Leader: 1017 Replicas: 1017,1018,1016 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 1 Leader: 1018 Replicas: 1018,1016,1017 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 2 Leader: 1016 Replicas: 1016,1017,1018 Isr: 1016,1018,1017 Topic: mno.de.pola.trump Partition: 3 Leader: 1017 Replicas: 1017,1016,1018 Isr: 1017,1018,1016 Topic: mno.de.pola.trump Partition: 4 Leader: 1018 Replicas: 1018,1017,1016 Isr: 1018,1017,1016 Topic: mno.de.pola.trump Partition: 5 Leader: 1016 Replicas: 1016,1018,1017 Isr: 1016,1018,1017 ... View more

hi all we run the following command on kafka machine in order to view/print the topics names we can see that its fails on Caused by: java.net.BindException: Address already in use but without to print also the port number ./kafka-topics.sh --zookeeper zookeeper1:2181 --list To enable GC log rotation, use -Xloggc:<filename> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=<num_of_files> where num_of_file > 0 GC log rotation is turned off Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386) at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401) Caused by: java.net.BindException: Address already in use any suggestion how to find the port ? that is use ... View more

hi all we want to verify the name nodes status from ambari by API ( namenode adtive/stand by under hdfs ) capture.png is it possible to create API for that? ... View more