Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka consumer groups are not getting listed and described with kerberos enabled

Highlighted

Kafka consumer groups are not getting listed and described with kerberos enabled

New Contributor

Hi,


I am using kafka version 0.10.1 on HDP 2.6 with kerberos enabled. When I am trying to get the list of consumer groups and describe group with below commands I am getting error message. Please advise.



[kafka@XXX ~]$ /usr/hdp/current/kafka-broker/bin/kafka-consumer-groups.sh --bootstrap-server xxxx:6667,xxxx:6667,xxxx:6667 --list --command-config /tmp/grouprop.properties

[2019-08-13 02:40:56,791] WARN TGT renewal thread has been interrupted and will exit. (org.apache.kafka.common.security.kerberos.KerberosLogin)


[kafka@XXX ]$ /usr/hdp/current/kafka-broker/bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server xxxx:6667 --describe --group uat_customer --command-config /tmp/grouprop.properties

Error while executing consumer group command Failed to send request after 5000 ms.

org.apache.kafka.common.errors.TimeoutException: Failed to send request after 5000 ms.


[2019-08-14 09:57:39,346] WARN TGT renewal thread has been interrupted and will exit. (org.apache.kafka.common.security.kerberos.KerberosLogin)



[kafka@XXX]$ /usr/hdp/current/kafka-broker/bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server xxxx:6667 --list --command-config /tmp/grouprop.properties

Error while executing consumer group command Request METADATA failed on brokers List(xxx:6667 (id: -1 rack: null))

java.lang.RuntimeException: Request METADATA failed on brokers List(xxxx:6667 (id: -1 rack: null))

at kafka.admin.AdminClient.sendAnyNode(AdminClient.scala:67)

at kafka.admin.AdminClient.findAllBrokers(AdminClient.scala:87)

at kafka.admin.AdminClient.listAllGroups(AdminClient.scala:96)

at kafka.admin.AdminClient.listAllGroupsFlattened(AdminClient.scala:117)

at kafka.admin.AdminClient.listAllConsumerGroupsFlattened(AdminClient.scala:121)

at kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.list(ConsumerGroupCommand.scala:304)

at kafka.admin.ConsumerGroupCommand$.main(ConsumerGroupCommand.scala:66)

at kafka.admin.ConsumerGroupCommand.main(ConsumerGroupCommand.scala)


[2019-08-14 10:03:05,877] WARN TGT renewal thread has been interrupted and will exit. (org.apache.kafka.common.security.kerberos.KerberosLogin)




cat /tmp/grouprop.properties

security.protocol=PLAINTEXTSASL


2 REPLIES 2

Re: Kafka consumer groups are not getting listed and described with kerberos enabled

Cloudera Employee

Hi Saurav, 

Can you please enable client DEBUG and see if the security.protocol is passed properly? also, it may help you also to get more detailed information about the timeout issue.

 

This can be done by editing below file:

 

/etc/kafka/conf/tools-log4j.properties

 

Change the following line from:

 

log4j.rootLogger=WARN, stderr

To

log4j.rootLogger=DEBUG, stderr

 

After that, run the consumer again and let us know how it goes.

Thanks,

Manuel.

Re: Kafka consumer groups are not getting listed and described with kerberos enabled

Cloudera Employee

Hi @sauravsuman689 

A common issue that people have when using the kafka-consumer-group command line tool is that they do not set it up to communicate over Kerberos like any other Kafka client (i.e. consumers and producers).

The security.protocol output you shared based on the cat command doesn't look right:

 

 

cat /tmp/grouprop.properties

security.protocol=PLAINTEXTSASL

 


This should instead be:

 

security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka

 


You can use the same instructions outlined in the following link starting with step number 5:
https://www.cloudera.com/documentation/kafka/latest/topics/kafka_security.html#concept_lcn_4mm_s5

I understand you're using HDP but it should be pretty much the same steps. You will of course just use the same command line tool command you're using as opposed to the consumer command mentioned in the link:

 

[kafka@XXX ~]$ /usr/hdp/current/kafka-broker/bin/kafka-consumer-groups.sh --bootstrap-server xxxx:6667,xxxx:6667,xxxx:6667 --list --command-config /tmp/grouprop.properties


EDIT:

It seems like HDP works a bit differently so your security.protocol parameter aligns with what the HDP platform would expect.