Produce Message in Kafka running in HDF 3.0.2 using Java Program

[#1]

Hello,

I have created "my-topic" in kafka and trying to produce message using Java program running on my local machine, outside of the HDF 3.0.2.

Kafka broker is running on port 6667 and the IP address of VMware box is 172.17.0.2

In my java program, bootstrap server config is as shown below.

Please let me know if any I am missing anything to produce message in kafka in HDF3.0.2 from a local machine.

Thanks much.

props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "172.17.0.2:6667");

[#2]

@Hiren Dossani

The code snippet looks good. What error or exception are you facing while executing the code?

If you are running the code on your local machine and trying to connect to the kafka broker on VMWare Box then please make sure that you are able to establish connection from your local box first like:

# telnet 172.17.0.2 6667

.

If not then please check if the Firewall is disabled or not? and if required check the port forwarding to see if you are allowed to access that port from your local machine.

[#3]

Thanks Jay for your reply. I disabled the firewall on my mac, yet no luck. No, I am not able to connect using telnet 172.17.0.2 6667

So far, I updated /etc/hosts file as 172.17.0.2 sandbox-hdf.hortonworks.com and updated java code to use sandbox-hdf.hortonworks.com in bootstrap server config but no success.

Although, I am able to list the topics using terminal window with below command:

bin/kafka-topics.sh --list --zookeeper 192.168.71.133:2181

Interchanging 172.17.0.2 to 192.168.71.133 in java program doesn't help either.

[#4]

@Hiren Dossani

Please check your HDP Sandbox settings to figure out if you have the Port Forwarding rule added to expose 6667 port?

https://community.hortonworks.com/articles/65914/how-to-add-ports-to-the-hdp-25-virtualbox-sandbox.h...

https://hortonworks.com/tutorial/sandbox-port-forwarding-guide/section/1/

.

[#5]

I have the HDF_3.0.2 running on VmWare.

Cannot even ssh into Vmware box using command ssh -p 2122 root@sandbox-hdf.hortonworks.com

Is there any simple example/steps to follow ? Thanks for your help and earliest attention.

I want to demo integrating jenkins with java and produce messages in Kafka in HDF using java from my local machine, outside of VmWare.

Thanks for your earliest help and attention.

[#6]

@Hiren Dossani

You are not able to do ssh into Vmware box using command because the port seems to be wrong for HDF ssh -p 2122 root@sandbox-hdf.hortonworks.com

.

For Doing SSH to HDF you will need to use a slightly different port. To login to the VM you will need to use 12122 port

# ssh root@127.0.0.1 -p 12122
Enter Password: hadoop

.

In order to login to the Docker Container where the HDF components are actually running you will need to use:

# ssh root@127.0.0.1 -p 12222
Enter Password: hadoop

.