question Re: Kafka client code does not currently support obtaining a password from the user in Support Questions

Kafka client code does not currently support obtaining a password from the user

sampathkumar_ma — Sat, 23 Nov 2019 14:40:21 GMT

I'm getting below error message while trying to produce data from kafka topic in the kerberized HDP cluster.

Error:

DEBUG [Producer clientId=console-producer] Kafka producer has been closed (org.apache.kafka.clients.producer.KafkaProducer)
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:457)
Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user

Stack:

HDP 3.1.0 Kafka 1.0.0.3.1

$KAFKA_HOME="/usr/hdp/3.1.0.0-78/kafka" $BROKER_LIST="<broker-list>" $ZK_HOSTS="<zk-host-list>:2181/kafka" $export KAFKA_OPTS="-Djava.security.auth.login.config=/home/<user>/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=true -Dsun.security.krb5.debug=true" $export KAFKA_CLIENT_KERBEROS_PARAMS="-Djava.security.auth.login.config=/home/<user>/jaas.conf -Dsun.security.krb5.debug=true"

$cat jaas.conf

---using user keytab & principal for authentication and disabled useTicketCache---

KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/home/<user>/user.keytab" storeKey=true useTicketCache=false serviceName="kafka" principal="user@domain.COM"; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/home/<user>/user.keytab" storeKey=true useTicketCache=false serviceName="zookeeper" principal="user@domain.COM"; };

$cat client.properties

security.protocol=SASL_PLAINTEXT sasl.mechanism=GSSAPI sasl.kerberos.service.name=kafka

$klist

~]$ klist klist: No credentials cache found (filename: /tmp/krb5cc_121852)

$kafka-console-producer.sh

$KAFKA_HOME/bin/kafka-console-producer.sh --broker-list <broker-list>:9092 --topic testtopic --producer.config /home/<user>/client.properties

full error log:

[2019-11-23 10:05:45,614] DEBUG Added sensor with name bufferpool-wait-time (org.apache.kafka.common.metrics.Metrics)
[2019-11-23 10:05:45,617] DEBUG Added sensor with name buffer-exhausted-records (org.apache.kafka.common.metrics.Metrics)
[2019-11-23 10:05:45,620] DEBUG Updated cluster metadata version 1 to Cluster(id = null, nodes = [sl975iaehdp0401.visa.com:9092 (id: -1 rack: null)], partitions = [], controller = null) (org.apache.kafka.clients.Metadata)
[2019-11-23 10:05:45,637] INFO [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 0 ms. (org.apache.kafka.clients.producer.KafkaProducer)
[2019-11-23 10:05:45,638] DEBUG [Producer clientId=console-producer] Kafka producer has been closed (org.apache.kafka.clients.producer.KafkaProducer)
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:457)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:304)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:45)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:153)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:140)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:414)
... 3 more
Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user
at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940)
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:60)
at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:103)
at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:65)
at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:125)
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:142)
... 7 more

Could you please help on this.

Thank you.

Re: Kafka client code does not currently support obtaining a password from the user

ManuelCalvo — Mon, 25 Nov 2019 17:01:07 GMT

@sampathkumar_ma

Could you please check if the user running the command has permissions to get a valid ticket from: "/home/<user>/user.keytab"

Also, add "debug=true" in the jaas file to get more details:

KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/home/<user>/user.keytab" storeKey=true debug=true useTicketCache=false serviceName="kafka" principal="user@domain.COM"; };

Re: Kafka client code does not currently support obtaining a password from the user

sampathkumar_ma — Mon, 25 Nov 2019 20:16:04 GMT

Hi @ManuelCalvo ,

Yes, keytab has right permission to get the valid ticket. I tried taking the ticket manually & it works fine.

What I observed here was environment variable KAFKA_OPTS was ignored by kafka clients.The console producer/consumer should work with the KAFKA_OPTS environment variable that is expected to have priority over the system variables;

I exported KAFKA_OPTS pointing to the JAAS file and Kerberos client configuration file, but it's not working!!!!!

Kafka-version : 2.0.0.3

export KAFKA_OPTS="-Djava.security.auth.login.config=/home/<user>/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf"

error:

Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:153)

If I pass SASL parameters as below in the client.properties, I'm able to produce/consume data from Topics without any issue.

$KAFKA_HOME/bin/kafka-console-producer.sh --broker-list $BROKER_LIST --topic testtopic --producer.config /home/<user>/client.properties

$cat client.properties security.protocol=SASL_PLAINTEXT sasl.mechanism=GSSAPI #sasl.kerberos.service.name=kafka sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \ useKeyTab=true \ storeKey=true \ keyTab="/home/<user>/<user>.keytab" \ useTicketCache=false \ serviceName="kafka" \ principal="user@domain.COM";

Any idea why export KAFKA_OPTS is not working here?

Thank you

Re: Kafka client code does not currently support obtaining a password from the user

ManuelCalvo — Tue, 26 Nov 2019 15:10:55 GMT

@sampathkumar_ma

export KAFA_OPTS should work in this case. Could you please add "debug=true" to the jaas file:

Share the complete output, we should see something similar to:

Debug is true storeKey false useTicketCache true useKeyTab false doNotPrompt false ticketCache is null isInitiator true KeyTab is null refreshKrb5Config is false principal is null tryFirstPass is false useFirstPass is false storePass is false clearPass is false Acquire TGT from Cache Principal is kafka/host@EXAMPLE.COM Commit Succeeded

Also along with that you can enable DEBUG under:

/etc/kafka/conf/tools-log4j.properties

Change WARN to DEBUG and run the client and share the details.

Re: Kafka client code does not currently support obtaining a password from the user

sampathkumar_ma — Sat, 30 Nov 2019 13:05:42 GMT

@ManuelCalvo

Changed WARN to DEBUG and ran the kafka producer. Please find the details below:

jaas.conf

KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/home/user.keytab" storeKey=true useTicketCache=false debug=true serviceName="kafka" principal="user@domain.COM"; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/home/<user>/user.keytab" storeKey=true useTicketCache=false debug=true serviceName="zookeeper" principal="user@domain.COM"; };

client.propertis

security.protocol=SASL_PLAINTEXT sasl.kerberos.service.name=kafka

kafka-producer:

[<user>@server ~]$export KAFKA_OPTS="-Djava.security.auth.login.config=/home/<user>/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true" [<user>@server ~]$ $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list $BROKER_LIST --producer.config /home/<user>/client.properties --topic testtopic

full error log:

[<user>@server ~]$ $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list $BROKER_LIST --producer.config /home/<user>/client.properties --topic testtopic [2019-11-30 12:52:57,917] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) [2019-11-30 12:52:57,977] INFO ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [server1:9092, server2:9092, server3:9092, server4:9092] buffer.memory = 33554432 client.id = console-producer compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = [] key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer linger.ms = 1000 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 1000 reconnect.backoff.ms = 50 request.timeout.ms = 1500 retries = 3 retry.backoff.ms = 100 sasl.client.callback.handler.class = null sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = kafka sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.login.callback.handler.class = null sasl.login.class = null sasl.login.refresh.buffer.seconds = 300 sasl.login.refresh.min.period.seconds = 60 sasl.login.refresh.window.factor = 0.8 sasl.login.refresh.window.jitter = 0.05 sasl.mechanism = GSSAPI security.protocol = SASL_PLAINTEXT send.buffer.bytes = 102400 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer (org.apache.kafka.clients.producer.ProducerConfig) [2019-11-30 12:52:57,997] DEBUG Added sensor with name bufferpool-wait-time (org.apache.kafka.common.metrics.Metrics) [2019-11-30 12:52:58,000] DEBUG Added sensor with name buffer-exhausted-records (org.apache.kafka.common.metrics.Metrics) [2019-11-30 12:52:58,191] DEBUG Updated cluster metadata version 1 to Cluster(id = null, nodes = [server1:9092 (id: -2 rack: null), server2:9092 (id: -1 rack: null), server3:9092 (id: -4 rack: null), server4:9092 (id: -3 rack: null)], partitions = [], controller = null) (org.apache.kafka.clients.Metadata) [2019-11-30 12:52:58,210] INFO [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 0 ms. (org.apache.kafka.clients.producer.KafkaProducer) [2019-11-30 12:52:58,211] DEBUG [Producer clientId=console-producer] Kafka producer has been closed (org.apache.kafka.clients.producer.KafkaProducer) org.apache.kafka.common.KafkaException: Failed to construct kafka producer at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:457) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:304) at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:45) at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala) Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:153) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:140) at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:414) ... 3 more Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:60) at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:103) at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:65) at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:125) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:142) ... 7 more [<user>@server ~]$

Re: Kafka client code does not currently support obtaining a password from the user

BMODEV — Mon, 15 Jun 2020 16:01:53 GMT

I am facing the same issue. Did any of you get this working ?

Re: Kafka client code does not currently support obtaining a password from the user

ManuelCalvo — Mon, 15 Jun 2020 16:37:30 GMT

From jaas file I see that the debug=true was added, on the other hand, the debug is not showing up in the producer output, which means that the jaas file provided is not picker up properly.

If you check the kafka-console-producer.sh you'll notice below lines:

# check if kafka_jaas.conf in config , only enable client_kerberos_params in secure mode. KAFKA_HOME="$(dirname $(cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ))" KAFKA_JAAS_CONF=$KAFKA_HOME/config/kafka_jaas.conf if [ -f $KAFKA_JAAS_CONF ]; then export KAFKA_CLIENT_KERBEROS_PARAMS="-Djava.security.auth.login.config=$KAFKA_HOME/config/kafka_client_jaas.conf" fi

Try editing kafka_client_jaas.conf or also you can try to export using KAFKA_CLIENT_KERBEROS_PARAMS and see if that helps.

Regards,

Manuel.