Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka Producer doesnt send messages...

Highlighted

Kafka Producer doesnt send messages...

Explorer

The code is running. It is almost everything Ok. I suppose it pass by Kerberos, as you can see below:

>>>KinitOptions cache name is /tmp/krb5cc_0 >>>DEBUG client principal is root@KERBEROS.COM >>>DEBUG server principal iskrbtgt/KERBEROS.COM@KERBEROS.COM >>>DEBUG key type: 18 >>>DEBUG auth time: Sun Jan 29 04:30:11 EST 2017 >>>DEBUG start time: Sun Jan 29 04:30:11 EST 2017 >>>DEBUG end time: Mon Jan 30 04:29:38 EST 2017 >>>DEBUG renew_till time: null >>> CCacheInputStream: readFlags() FORWARDABLE; PROXIABLE; INITIAL; >>>DEBUG client principal is root@KERBEROS.COM >>>DEBUG server principal is X-CACHECONF: /krb5_ccache_conf_data/fast_avail/krbtgt/KERBEROS.COM@KERBEROS.COM@KERBEROS.COM >>>DEBUG key type: 0 >>>DEBUG auth time: Wed Dec 31 19:00:00 EST 1969 >>>DEBUG start time: null >>>DEBUG end time: Wed Dec 31 19:00:00 EST 1969 >>>DEBUG renew_till time: null >>> CCacheInputStream: readFlags() >>> unsupported key type found the default TGT: 18 >>> KeyTabInputStream, readName(): KERBEROS.COM >>> KeyTabInputStream, readName(): root >>> KeyTab: load() entry length: 51; type: 23 Looking for keys for: root@KERBEROS.COM Added key: 23version: 2 >>> KdcAccessibility: reset Looking for keys for: root@KERBEROS.COMAdded key: 23version: 2 Using builtin default etypes for default_tkt_enctypes default etypes for default_tkt_enctypes: 17 16 23. >>> KrbAsReq creating message >>> KrbKdcReq send: kdc=localhost UDP:88, timeout=30000, number of retries =3, #bytes=132 >>> KDCCommunication: kdc=localhost UDP:88, timeout=30000,Attempt =1, #bytes=132 >>> KrbKdcReq send: #bytes read=618 >>> KdcAccessibility: remove localhost Looking for keys for: root@KERBEROS.COM Added key: 23version: 2 >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType >>> KrbAsRep cons in KrbAsReq.getReply root

But the message doesnt reach the topic.

Could you help me?

Thanks!

3 REPLIES 3
Highlighted

Re: Kafka Producer doesnt send messages...

Super Mentor

@Rafael Menezes

Above messages are mostly "DEBUG" messages. Are you noticing any error / exception in your code execution?

Is your kafka producer really getting connected to the broker? Do you see any new log message at kafka side as soon as you run the Kafka Producer code?

Also as per the following link there are some "Kafka Client" side configurations are needed as well for a kerberized environment (Have you already done that)

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_secure-kafka-ambari/content/ch_secure-kaf...

JAAS Configuration File for the Kafka Client
----------------------------------------------
Kafka client configuration with keytab, for producers:
KafkaClient {
     com.sun.security.auth.module.Krb5LoginModule required
     useKeyTab=true
     keyTab="/etc/security/keytabs/storm.service.keytab"
     storeKey=true
     useTicketCache=false
     serviceName="kafka"
     principal="storm@EXAMPLE.COM";
    };Kafka client configuration without keytab, for producers:
KafkaClient {
     com.sun.security.auth.module.Krb5LoginModule required
     useTicketCache=true
     renewTicket=true
     serviceName="kafka";
    };Kafka client configuration for consumers:
 KafkaClient {
     com.sun.security.auth.module.Krb5LoginModule required
     useTicketCache=true
     renewTicket=true
     serviceName="kafka";
     };

.

Highlighted

Re: Kafka Producer doesnt send messages...

Super Mentor

@Rafael Menezes

Additionally the following link might be useful while writing the Kafka Producer code : http://henning.kropponline.de/2016/02/21/secure-kafka-java-producer-with-kerberos/

props.put("security.protocol", "SASL_PLAINTEXT");
props.put("sasl.kerberos.service.name", "kafka");

.

Highlighted

Re: Kafka Producer doesnt send messages...

Explorer
@Jay SenSharma

I have been trying to build the producer to be connected to a common Kafka. After that, I would test this in a Hortonworks Kafka and Ambari. I think that way is simpler and I have had difficult to install Ambari in my computer.

Going back to the problema...the only message that is similar to a wrong thing is the following one:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details

Not sure if it is connected with Kafka...

Don't have an account?
Coming from Hortonworks? Activate your account here