Kafka multi broker setup through ambari

[#1]

i configure multibroker kafka through ambari but failed to startup the kafka broker. however, it was successful when tried doing it through command line (with diff protocol,diff portnos, diff nodes)

and now i'm very much confused about the listeners property because by default amabri bound to 6667 port, what should be used for other listeners if im configuring multibroker.

all most i tried all the combinations but non of them helped me while starting it from ambari and it gives me unable to connect to zookeeoer.

example :

1. ( diff protocol, diff portnos, diff nodes) - this worked from command line but not through ambari

2.(diff protocol, diff portnos,same node) - this worked from command line but not through ambari

3. its not working neither command line nor ambari if this has been set to 'same protocol' or 'same port' -

please help me

sathish

[#2]

could you please post the kafka broker startup logs which is failing after setting 'same protocol' and same port.

[#3]

below are my configurations

listeners = PLAINTEXTSASL://10.246.131.99:6667, PLAINTEXTSASL://10.246.131.34:6667, PLAINTEXTSASL://10.246.131.35:6667

advertised.listeners = PLAINTEXTSASL://10.246.131.99:6667, PLAINTEXTSASL://10.246.131.34:6667, PLAINTEXTSASL://10.246.131.35:6667

below is the error

[2016-12-19 05:57:09,852] FATAL (kafka.Kafka$)

java.lang.IllegalArgumentException: requirement failed: Each listener must have a different port

at scala.Predef$.require(Predef.scala:233)

at kafka.server.KafkaConfig.validateUniquePortAndProtocol(KafkaConfig.scala:906)

at kafka.server.KafkaConfig.getListeners(KafkaConfig.scala:914)

at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:867)

at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:699)

at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:696)

at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)

at kafka.Kafka$.main(Kafka.scala:58)

at kafka.Kafka.main(Kafka.scala)

[#4]

@sathish jeganathan

could you please try following steps

1. open ambari, open kafka configuration

2. update listeners=SASL_PLAINTEXT://localhost:6667

3.remove advartised.listeners and save configuration

4. restart kafka broker and see if helps

[#5]

@sathish jeganathan any luck with the above mentioned settings.

[#6]

Hi Raj - ive tried removing advertised listeners but it gets generated automatically.. also still am getting same error "Each listener must have a different port"

thanks,

sathish

[#7]

also i tried with diff host name with same protocol and i got below error

Each listener must have a different protocol

java.lang.IllegalArgumentException: requirement failed: Each listener must have a different protocol

at scala.Predef$.require(Predef.scala:233)

at kafka.server.KafkaConfig.validateUniquePortAndProtocol(KafkaConfig.scala:907)

at kafka.server.KafkaConfig.getListeners(KafkaConfig.scala:914)

at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:867)

at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:699)

at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:696)

at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)

at kafka.Kafka$.main(Kafka.scala:58)

at kafka.Kafka.main(Kafka.scala)

[#8]

and now ive changed all the protocols,port nos (9091,92,93) and started kafka through ambari but it started giving me the below error

FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)

org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 25000

thanks,

sathish

[#9]

@sathish jeganathanthere is no need to specify advertised listener,what hdp version you are currently running with? can you try to set following setting and see if it helps.make both listeners and advartised.listeners same.

listeners=SASL_PLAINTEXT://localhost:6667

advartised.listeners=SASL_PLAINTEXT://localhost:6667

[#10]

@Rajkumar Singh

there is no parameter advertised.listeners in ambari-kafka config but it exists in server.properties. however advertised.listeners property has got the same value as "listeners" ( both has got same values).

thanks,

sathish