Member since
08-29-2018
4
Posts
0
Kudos Received
0
Solutions
08-30-2018
04:46 AM
poojary_sudth: Thank you, but that is +- the same set of tasks I did before. I found how to make consumer work, it was necessary to add parameter --partition 0: KAFKA_OPTS="-Djava.security.auth.login.config=/root/jaas.conf" kafka-console-consumer --bootstrap-server ourHost:9092 --topic test --consumer.config /root/client.properties --partition 0 I cannot see all the messages comming into the topic, but at least some of them which fall into specified partition are printed. Which is enought for me to confirm that Kafka broker works. I found this hint here: https://stackoverflow.com/questions/34844209/consumer-not-receiving-messages-kafka-console-new-consumer-api-kafka-0-9
... View more
08-30-2018
02:20 AM
Hi, I tried and it did not help, consumer still prints nothing. Could it be blocked by Sentry? We did not configure topic-permissions yet, however producer can write there. I also tried to consume from non-existing topic and in this case it properly shows Error with fetching metadata.
... View more
08-30-2018
01:57 AM
pdvorak: Thank you, it lead me to running producer and consumer without errors. I just modified configuration to unsecured 9092 port. jaas.conf:
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=false
useTicketCache=true
keyTab="somePathToKeytab"
principal="somePrincipal";
};
client.properties:
security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka
sasl.mechanism=GSSAPI
ssl.client.auth=none I can start producer and consumer with the following commands now: producer:
KAFKA_OPTS="-Djava.security.auth.login.config=/root/jaas.conf" kafka-console-producer --broker-list ourHost:9092 --topic test --producer.config /root/client.properties
consumer:
KAFKA_OPTS="-Djava.security.auth.login.config=/root/jaas.conf" kafka-console-consumer --bootstrap-server ourHost:9092 --topic test --consumer.config /root/client.properties Although, it does not throw any errors, consumer does not print any messages. Do you have any idea where might the problem be now? I can see number of sent messages in Cloudera Manager Chart "Total Messages Received Across Kafka Brokers". So, I assume they were sent properly by a producer.
... View more
08-29-2018
06:24 AM
I got exactly the same problem with the same question. ... 18/08/29 15:38:00 INFO utils.AppInfoParser: Kafka version : 1.0.1-kafka-3.1.0-SNAPSHOT 18/08/29 15:38:00 INFO utils.AppInfoParser: Kafka commitId : unknown >hello 18/08/29 15:38:13 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ourhost:9092 (id: -1 rack: null) disconnected ... My Kafka parcel version is: 3.1.0-1.3.1.0.p0.35
... View more