@ManuelCalvo  Changed WARN to DEBUG and ran the kafka producer.  Please find the details below: jaas.conf   KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/home/user.keytab" storeKey=true useTicketCache=false debug=true serviceName="kafka" principal="user@domain.COM"; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/home/<user>/user.keytab" storeKey=true useTicketCache=false debug=true serviceName="zookeeper" principal="user@domain.COM"; };   client.propertis   security.protocol=SASL_PLAINTEXT sasl.kerberos.service.name=kafka   kafka-producer:   [<user>@server ~]$export KAFKA_OPTS="-Djava.security.auth.login.config=/home/<user>/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true" [<user>@server ~]$ $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list $BROKER_LIST --producer.config /home/<user>/client.properties --topic testtopic   full error log:   [<user>@server ~]$ $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list $BROKER_LIST --producer.config /home/<user>/client.properties --topic testtopic [2019-11-30 12:52:57,917] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) [2019-11-30 12:52:57,977] INFO ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [server1:9092, server2:9092, server3:9092, server4:9092] buffer.memory = 33554432 client.id = console-producer compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = [] key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer linger.ms = 1000 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 1000 reconnect.backoff.ms = 50 request.timeout.ms = 1500 retries = 3 retry.backoff.ms = 100 sasl.client.callback.handler.class = null sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = kafka sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.login.callback.handler.class = null sasl.login.class = null sasl.login.refresh.buffer.seconds = 300 sasl.login.refresh.min.period.seconds = 60 sasl.login.refresh.window.factor = 0.8 sasl.login.refresh.window.jitter = 0.05 sasl.mechanism = GSSAPI security.protocol = SASL_PLAINTEXT send.buffer.bytes = 102400 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer (org.apache.kafka.clients.producer.ProducerConfig) [2019-11-30 12:52:57,997] DEBUG Added sensor with name bufferpool-wait-time (org.apache.kafka.common.metrics.Metrics) [2019-11-30 12:52:58,000] DEBUG Added sensor with name buffer-exhausted-records (org.apache.kafka.common.metrics.Metrics) [2019-11-30 12:52:58,191] DEBUG Updated cluster metadata version 1 to Cluster(id = null, nodes = [server1:9092 (id: -2 rack: null), server2:9092 (id: -1 rack: null), server3:9092 (id: -4 rack: null), server4:9092 (id: -3 rack: null)], partitions = [], controller = null) (org.apache.kafka.clients.Metadata) [2019-11-30 12:52:58,210] INFO [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 0 ms. (org.apache.kafka.clients.producer.KafkaProducer) [2019-11-30 12:52:58,211] DEBUG [Producer clientId=console-producer] Kafka producer has been closed (org.apache.kafka.clients.producer.KafkaProducer) org.apache.kafka.common.KafkaException: Failed to construct kafka producer at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:457) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:304) at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:45) at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala) Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:153) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:140) at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:414) ... 3 more Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:60) at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:103) at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:65) at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:125) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:142) ... 7 more [<user>@server ~]$     ... View more

Hi @ManuelCalvo ,   Yes, keytab has right permission to get the valid ticket. I tried taking the ticket manually & it works fine.   What I observed here was environment variable KAFKA_OPTS was ignored by kafka clients.The console producer/consumer should work with the KAFKA_OPTS environment variable that is expected to have priority over the system variables;    I exported KAFKA_OPTS pointing to the JAAS file and Kerberos client configuration file, but it's not working!!!!! Kafka-version : 2.0.0.3   export KAFKA_OPTS="-Djava.security.auth.login.config=/home/<user>/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf"   error:  Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:153) If I pass SASL parameters as below in the client.properties, I'm able to produce/consume data from Topics without any issue.   $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list $BROKER_LIST --topic testtopic --producer.config /home/<user>/client.properties $cat client.properties security.protocol=SASL_PLAINTEXT sasl.mechanism=GSSAPI #sasl.kerberos.service.name=kafka sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \ useKeyTab=true \ storeKey=true \ keyTab="/home/<user>/<user>.keytab" \ useTicketCache=false \ serviceName="kafka" \ principal="user@domain.COM";    Any idea why export  KAFKA_OPTS is not working here? Thank you ... View more

okay sure @Geoffrey Shelton Okot, will talk to AD team on this and let you know the status. Thanks. ... View more

It's just a workaround @Geoffrey Shelton Okot. Thanks. ... View more

Hi @Geoffrey Shelton Okot, Thanks for your time. I have set the below two properties in core-site.xml from Ambari. Now, NN, RM and History server UI is working fine. hadoop.http.authentication.simple.anonymous.allowed=true hadoop.http.authentication.type=simple Regards, Sampath ... View more

Hi @Geoffrey Shelton Okot, Thanks for the response. I have updated the krb5.conf with the below properties # grep "enctypes" /etc/krb5.conf default_tgs_enctypes= des3-cbc-sha1 aes256-cts-hmac-sha1-96 arcfour-hmac aes128-cts-hmac-sha1-96 des-cbc-md5 default_tkt_enctypes = des3-cbc-sha1 aes256-cts-hmac-sha1-96 arcfour-hmac aes128-cts-hmac-sha1-96 des-cbc-md5 # klist -aef Ticket cache: FILE:/tmp/krb5cc_0 Default principal: HTTP/hostname_fqdn@realm Valid starting Expires Service principal 02/09/2019 14:44:22 02/10/2019 00:44:22 krbtgt/realm@realm renew until 02/16/2019 14:44:22, Flags: FRIA Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96 Addresses: (none) I don't have access to check the encryption types mapped in AD server. Is there any way I can check this from my linux host? Thank you. ... View more

Hi @Andreas Kühnert, Could you refer the below thread. Hope this helps! https://community.hortonworks.com/questions/224670/yarn-registry-dns-start-failed-hortonworks-3.html ... View more