question Re: kafka console producer for file message in Support Questions

kafka console producer for file message

uday123 — Wed, 04 Oct 2023 12:16:25 GMT

kafka version cdh 6.3.4

console producer for single message from command line works fine but issue with below command

console producer command :

1. kafka-console-producer --broker-list <bokerhost:port> --topic <my-topic> --producer.config source-broker-1.properties < sampleMsg.txt

2. cat sampleMsg.txt | kafka-console-producer --broker-list <bokerhost:port> --topic <my-topic> --producer.config source-broker-1.properties

3. echo sampleMsg.txt | kafka-console-producer --broker-list <bokerhost:port> --topic <my-topic> --producer.config source-broker-1.properties

my property file looks like:

security.protocol=SASL_SSL
sasl.kerberos.service.name=kafka
ssl.truststore.location=/cloudera/security/jks/jks file

and set jaas.conf correctly

Re: kafka console producer for file message

cjervis — Wed, 04 Oct 2023 19:12:39 GMT

Welcome to the community @uday123. While you wait for someone with more experience to answer, have you looked over this in case it may get you closer?

Configuring Kafka Console Producer/Consumer on a Secured CDP Cluster

Re: kafka console producer for file message

uday123 — Wed, 04 Oct 2023 19:18:00 GMT

Thanks @cjervis .

Mentioned console producer command from link is working for me however the syntax or command not working when I am passing message from file. I am able to produce message from console itself but not from file and seeking help for that

Re: kafka console producer for file message

DianaTorres — Wed, 04 Oct 2023 20:16:49 GMT

@rki_ @ChethanYM Do you have any insights here? Thanks!

Re: kafka console producer for file message

arandive — Tue, 10 Oct 2023 10:30:07 GMT

Hi @uday123
Do you see any error/warning messages when you hit the command?
I understood that the console producer for a single message from the command line works for you but did you try to consume the same messages using the console consumer? If yes, do you see the same messages?

- It could be that the console producer for a single message from the command line might not work and it is just that you are not seeing any warning/error message as you could be interrupting the console producer before it is getting timed out exception. I would request you to kindly check and confirm if you can able to produce as well as consume the same messages.

$ Points to check for,
1. Check if you have a valid ticket before you run the command.
2. Export the JAAS.CONF file.
3. Make sure the user keytab/principal that you are using has enough privileges in the ranger-kafka policy to produce the messages to the problematic topic.
4. Check if the file 'source-broker-1.properties' has read access to the user.
5. Since you are using the SASL_SSL protocol, check and use the same port number that is assigned to SASL_SSL.

Re: kafka console producer for file message

DianaTorres — Fri, 13 Oct 2023 18:12:10 GMT

@uday123 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.