Kerberized HDF PublishKafka_10_0 to Kerberized HDP

colton_rodgers — Tue, 21 Apr 2026 12:25:48 GMT

@kkawamura @Matt @Ali Bajwa @Bryan Bende

I am continually running into the following error and would be extremely grateful for any help. Please let me know if there are any more files or configurations of which you would like to see.

javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner  authentication information from the user

Both HDF and HDP are Kerberized and running off of the same KDC. HDF and HDP both have Kafka installed, but only the HDP host with Kafka is used in the configurations and the keytabs.I have followed the steps shown in just about every post I could find in order to relieve this issue, but still to no avail. Other pertinent config files will be listed below: On HDF: kafka-jaas.conf

Client {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/etc/security/keytabs/nifi.service.keytab"
   useTicketCache=false
   principal="nifi/{_HOST}@ZTEST.LOCAL"
};


KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=false
renewTicket=true
serviceName="kafka"
useKeyTab=true
keyTab="/etc/security/keytabs/kafka1.service.keytab"
principal="kafka/{_HOST}@ZTEST.LOCAL";
};

bootstrap.conf

# Java command to use when running NiFi
java=java

# Username to use when running NiFi. This value will be ignored on Windows.
run.as=nifi
##run.as=root

# Configure where NiFi's lib and conf directories live
lib.dir=/usr/hdf/2.0.2.0-17/nifi/lib
conf.dir=/usr/hdf/2.0.2.0-17/nifi/conf

# How long to wait after telling NiFi to shutdown before explicitly killing the Process
graceful.shutdown.seconds=20

# Disable JSR 199 so that we can use JSP's without running a JDK
java.arg.1=-Dorg.apache.jasper.compiler.disablejsr199=true

# JVM memory settings
java.arg.2=-Xms512m
java.arg.3=-Xmx512m

# Enable Remote Debugging
#java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000

java.arg.4=-Djava.net.preferIPv4Stack=true

# allowRestrictedHeaders is required for Cluster/Node communications to work properly
java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true
java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol

# The G1GC is still considered experimental but has proven to be very advantageous in providing great
# performance without significant "stop-the-world" delays.
java.arg.13=-XX:+UseG1GC

#Set headless mode by default
java.arg.14=-Djava.awt.headless=true

#Ambari Metrics Collector URL - passed in to flow.xml for AmbariReportingTask
java.arg.15=-Dambari.metrics.collector.url=http://{_HOST}:6188/ws/v1/timeline/metrics

#Application ID - used in flow.xml - passed into flow.xml for AmbariReportingTask
java.arg.16=-Dambari.application.id=nifi

java.arg.17=-Djava.security.auth.login.config=/etc/nifi/kafka-jaas.conf

PublishKafka_10_0

Kafka Brokers: {_HOST}:6667Security Protocol: SASL_PLAINTEXTKerberos Service Name: kafka

On HDP: kafka_jaas.conf

KafkaServer {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/etc/security/keytabs/kafka.service.keytab"
   storeKey=true
   useTicketCache=false
   serviceName="kafka"
   principal="kafka/{_HOST}@ZTEST.LOCAL";
};
KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useTicketCache=true
   renewTicket=true
   serviceName="kafka";
};
Client {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/etc/security/keytabs/kafka.service.keytab"
   storeKey=true
   useTicketCache=false
   serviceName="zookeeper"
   principal="kafka/{_HOST}@ZTEST.LOCAL";
};

Thank you in advance for any help! **Please note all hostnames were replaced with {_HOST}Full Stacktrace from nifi-app.log:

2017-01-12 11:18:42,157 WARN [Timer-Driven Process Thread-10] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335) ~[na:na]
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:163) ~[na:na]
        at org.apache.nifi.processors.kafka.pubsub.PublisherPool.createLease(PublisherPool.java:61) ~[na:na]
        at org.apache.nifi.processors.kafka.pubsub.PublisherPool.obtainPublisher(PublisherPool.java:56) ~[na:na]
        at org.apache.nifi.processors.kafka.pubsub.PublishKafka_0_10.onTrigger(PublishKafka_0_10.java:312) ~[na:na]
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) ~[nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17
]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_77]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_77]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_77]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_77]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner  authentication information from the user
        at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86) ~[na:na]
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:71) ~[na:na]
        at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83) ~[na:na]
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:277) ~[na:na]
        ... 16 common frames omitted
Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner  authentication information from the user
        at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940) ~[na:1.8.0_77]
        at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760) ~[na:1.8.0_77]
        at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) ~[na:1.8.0_77]
        at sun.reflect.GeneratedMethodAccessor368.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) ~[na:1.8.0_77]
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) ~[na:1.8.0_77]
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) ~[na:1.8.0_77]
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) ~[na:1.8.0_77]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_77]
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) ~[na:1.8.0_77]
        at javax.security.auth.login.LoginContext.login(LoginContext.java:587) ~[na:1.8.0_77]
        at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:69) ~[na:na]
        at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:110) ~[na:na]
        at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:46) ~[na:na]
        at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:68) ~[na:na]
        at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78) ~[na:na]
        ... 19 common frames omitted

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

bbende — Fri, 13 Jan 2017 00:31:48 GMT

This may be just be a typo on the post, but you said "On HDF: kafka_jaas.conf" and then in NiFi's bootstrap you have "/etc/nifi/kafka-jaas.conf" so is it kafka_jass.conf or kafka-jass.conf?

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

colton_rodgers — Fri, 13 Jan 2017 00:33:08 GMT

Typo, let me fix that up.

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

bbende — Fri, 13 Jan 2017 00:51:30 GMT

Alright I don't see anything wrong jumping out at me, this is what has worked for me:

https://community.hortonworks.com/questions/63180/error-in-nifi-flow.html#answer-63240

Usually when it gets as far as prompting for a password, it is because something with your keytab and principal was not correct, maybe try listing the keytab and making sure the principal you are using is actually in that keytab.

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

colton_rodgers — Fri, 13 Jan 2017 01:04:51 GMT

So far looking good, turns out had nifi running as admin user which did not have permissions on that keytab, so far so good, will update with solved once tests confirm. Thank you! For all searching: Make sure the user that nifi is running as has permissions to access the keytab file... Whoops.

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

colton_rodgers — Fri, 13 Jan 2017 01:05:06 GMT

For all searching: Make sure the user that nifi is running as has permissions to access the keytab file and if after this you are receiving a metadata timeout error, login to zkCli.sh with Kafka ticket in cache...

kinit kafka/_HOST@REALM.EXAMPLE -kt /etc/security/keytabs/kafka.service.keytab

and rmr /brokers Restart Kafka and everything should be good to go at that point.

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

colton_rodgers — Fri, 13 Jan 2017 01:12:08 GMT

Now getting error regarding timeout waiting for metadata, any idea here or would you rather I open a new question?

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

bbende — Fri, 13 Jan 2017 02:55:10 GMT

The issue about waiting for metadata likely means the machine running NiFi can't reach one of the Kafka brokers hosting the topic. There is some property in Kafka that I can't remember that controls what host/ip it presents to external clients which sometimes needs to be set.

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

colton_rodgers — Fri, 13 Jan 2017 02:59:02 GMT

Changed the FQDN to IP and everything is flowing well now, thank you again for your help!

question Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP in Archives of Support Questions (Read Only)

Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP

Re: Kerberized HDF PublishKafka_10_0 to Kerberized HDP