Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

kafka produce/consume api not working on windows when the kafka server running on linux

Highlighted

kafka produce/consume api not working on windows when the kafka server running on linux

Explorer

kafka produce and consume api only working from linux host. trying to connect through api from another window machine i cannot produce any data. note that i tried to configure the advertised listener as indicated in many blogs, and the telnet from the remote windows machine is working. but still i cannot produce not consume the messages. note that when i run the kafka server on windows and the produce/consumer api on linux it work normally.

9 REPLIES 9
Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Mentor

@bob rabih

With your current config set as is ,type the below commands, replace the value remote_machine with ip/hostname{FQDN} of your remote machine kafka broker. Make sure your DNS is working

Producer

kafka/bin/kafka-console-producer.sh \ 
--broker-list remote_machine:9092 \ 
--topic {your_topic} 

Consumer

kafka/bin/kafka-console-consumer.sh \ 
--bootstrap-server remote_machine:9092 \ 
--topic {your_topic} \ 
--from-beginning 

This should work please revert

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Explorer

sorry actually it is not working from windows host. i tried a linux(centos) host it worked normally. note that the kafka server is running on linux(centos) host @Geoffrey Shelton Okot

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Super Collaborator
@bob rabih

sh scripts are meant to be ran from *nix machines, not from Windows.

Use the appropriate bat files in the kafka\bin\windows folder.

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Explorer

i am saying that the api (consuming from java code)is not working, surly i used the .bat file to test cli and it didn't work. @Jordan Moore, @Geoffrey Shelton Okot

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Mentor

@bob rabih

As reiterated by Jordan use the appropriate OS script

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Mentor

@bob rabih

Can you share java code?

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Explorer

@Geoffrey Shelton Okot the usual code found on the net

System.out.println("start produce");
		Properties props = new Properties();
		props.put("bootstrap.servers", "192.168.137.130:9092");
		props.put("acks", "all");


		props.put("retries", 0);
		props.put("batch.size", 16384);
		props.put("linger.ms", 1);
		props.put("buffer.memory", 33554432);
		props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
		props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");


		Producer<String, String> producer = new KafkaProducer<String, String>(props);
		for (int i = 0; i < 800;i++)
		{
				System.out.println(i);
			String sentence = "{i=" + i+ " \"_id\" : { \"$oid\" : \"5a8eefc21013462f98eb78e3\" }, \"metadata\" : { \"airportid\" : 0.0 }, \"filename\" : \"1.jpg\", \"aliases\" : null, \"chunkSize\" : { \"$numberLong\" : \"261120\" }, \"uploadDate\" : { \"$date\" : 1519316930479 }, \"length\" : { \"$numberLong\" : \"1504413\" }, \"contentType\" : null, \"md5\" : \"28573cc502292a21ec0dfb6bea15d0fb\" }";
					
					
					


			producer.send(new ProducerRecord<String, String>("test1", sentence, sentence));


		}
		producer.close();
	}

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Explorer

@Geoffrey Shelton Okot also note that the cli from windows(using the .bat files) does not work

Highlighted

Re: kafka produce/consume api not working on windows when the kafka server running on linux

Explorer
running the server on windows and using the same code to produce from linux machine worked normally. could it be a security or ports issue@Geoffrey Shelton Okot
Don't have an account?
Coming from Hortonworks? Activate your account here