Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

kafka produce/consume api not working on windows when the kafka server running on linux

Explorer

kafka produce and consume api only working from linux host. trying to connect through api from another window machine i cannot produce any data. note that i tried to configure the advertised listener as indicated in many blogs, and the telnet from the remote windows machine is working. but still i cannot produce not consume the messages. note that when i run the kafka server on windows and the produce/consumer api on linux it work normally.

9 REPLIES 9

Mentor

@bob rabih

With your current config set as is ,type the below commands, replace the value remote_machine with ip/hostname{FQDN} of your remote machine kafka broker. Make sure your DNS is working

Producer

kafka/bin/kafka-console-producer.sh \ 
--broker-list remote_machine:9092 \ 
--topic {your_topic} 

Consumer

kafka/bin/kafka-console-consumer.sh \ 
--bootstrap-server remote_machine:9092 \ 
--topic {your_topic} \ 
--from-beginning 

This should work please revert

Explorer

sorry actually it is not working from windows host. i tried a linux(centos) host it worked normally. note that the kafka server is running on linux(centos) host @Geoffrey Shelton Okot

Super Collaborator
@bob rabih

sh scripts are meant to be ran from *nix machines, not from Windows.

Use the appropriate bat files in the kafka\bin\windows folder.

Explorer

i am saying that the api (consuming from java code)is not working, surly i used the .bat file to test cli and it didn't work. @Jordan Moore, @Geoffrey Shelton Okot

Mentor

@bob rabih

As reiterated by Jordan use the appropriate OS script

Mentor

@bob rabih

Can you share java code?

Explorer

@Geoffrey Shelton Okot the usual code found on the net

System.out.println("start produce");
		Properties props = new Properties();
		props.put("bootstrap.servers", "192.168.137.130:9092");
		props.put("acks", "all");


		props.put("retries", 0);
		props.put("batch.size", 16384);
		props.put("linger.ms", 1);
		props.put("buffer.memory", 33554432);
		props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
		props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");


		Producer<String, String> producer = new KafkaProducer<String, String>(props);
		for (int i = 0; i < 800;i++)
		{
				System.out.println(i);
			String sentence = "{i=" + i+ " \"_id\" : { \"$oid\" : \"5a8eefc21013462f98eb78e3\" }, \"metadata\" : { \"airportid\" : 0.0 }, \"filename\" : \"1.jpg\", \"aliases\" : null, \"chunkSize\" : { \"$numberLong\" : \"261120\" }, \"uploadDate\" : { \"$date\" : 1519316930479 }, \"length\" : { \"$numberLong\" : \"1504413\" }, \"contentType\" : null, \"md5\" : \"28573cc502292a21ec0dfb6bea15d0fb\" }";
					
					
					


			producer.send(new ProducerRecord<String, String>("test1", sentence, sentence));


		}
		producer.close();
	}

Explorer

@Geoffrey Shelton Okot also note that the cli from windows(using the .bat files) does not work

Explorer
running the server on windows and using the same code to produce from linux machine worked normally. could it be a security or ports issue@Geoffrey Shelton Okot
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.