Member since
01-26-2017
7
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
15297 | 02-24-2017 07:39 PM |
02-24-2017
07:39 PM
1 Kudo
Well, we have it working now. @jwhitmore What we ended up doing is creating a VM in virtualbox with a hostonly network adapter. Added the hostonly IP addr with mappings to the 'sandbox.hortonworks.com' FQDN to the hosts file in Windows for the local machine, added the port forwarding/binding to the container at 6667 - 6667, and it started working. Short answer, but overall it was the lack of mapping from the ip addr to the FQDN.
... View more
02-17-2017
01:07 AM
Hmmm, when I try to telnet from host machine at 127.0.01:6667 I get: Connection closed by remote host When I try to ssh from host machine at 127.0.0.1:6667 I get: server unexpectedly closed network connection ---------------------------------- Also tried these experiments from the virtual box with the container running and Kafka started telnet from virtual box telnet 127.0.0.1 6667
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'. and netstat -tan | grep :6667
tcp6 0 0 :::6667 :::* LISTEN
tcp6 0 0 :::8080 :::* LISTEN
tcp6 0 0 :::2222 :::* LISTEN
So it looks like the port forwarding is working between host and VM, and VM and container, but not between host machine and container? Of note, I still get the output between the VM and the Container even if the services at those ports are not running, however, the connection is automatically closed as opposed to the CLI waiting for me to input Ctrl-C. So I guess what this shows is that the port forwarding is working, however, I'm not sure it elucidates where the error with Producing to Kafka is, or why the host machine can't make the same connections as the VM with regards to the container. The Windows service configuration is: Kafka uri: http://127.0.0.1:6667 and when I run the service on my host machine and a consumer in the container, this is what I see: bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic test-topic {metadata.broker.list=sandbox.hortonworks.com:6667, request.timeout.ms=30000, client.id=console-consumer-59653, security.protocol=PLAINTEXT}
^CProcessed a total of 0 messages
But no errors on the host machine...
... View more
02-16-2017
07:36 PM
John, Thank you for the response. Yes, I am running the C# service and VirtualBox on the same windows host. Here is what I get with those two experiments. netstat -ano | findstr "6667 8080"
TCP 127.0.0.1:6667 0.0.0.0:0 LISTENING 3344
TCP 127.0.0.1:8080 0.0.0.0:0 LISTENING 3344
TCP 127.0.0.1:18080 0.0.0.0:0 LISTENING 3344
TCP [::1]:62738 [::1]:49154 ESTABLISHED 28080
and for the browser This site can’t be reached
The webpage at <a href="http://127.0.0.1:6667/">http://127.0.0.1:6667/</a> might be temporarily down or it may have moved permanently to a new web address.
ERR_UNSAFE_PORT @jwhitmore
... View more
02-16-2017
01:30 PM
2 Kudos
We are trying to produce to Kafka in the HDP 2.5 sandbox from my host machine, and I can't seem to get it to work. I am doing this from a C# service running in Visual Studio. It seems that the connection isn't being made between the host machine and the Kafka Borker host. We had this working before with HDP 2.4, but now that it is in a docker container, there seems to be another layer of complexity that is screwing us up. Here is what we have tried so far: - Added 6667 to 6667 between host and VM. Killed the sandbox container, removed it, added 6667:6667 between VM and Container to /root/start_scripts/start_sandbox.sh, restarted the VM, saw the port forwarding was added with docker ps. We then created the Kafka topic in the container, (tested produce and consume within container; which worked ok) and then tried to produce from the host machine with the following uri: http://127.0.0.1:6667. Doing it this way, I don't get an error message, it just doesn't show up on the consumer. Not sure what is happening there. - We also tried switching the system over to Host Only, but that seemed to screw with Ambari somehow, that is a different issue. - We also tried importing the image in our own OS environment in VBox, that didn't work either I don't mean post this question without a lot of detailed error messages and command line output, but there were just so many ways we have tried this that I wanted to see if anyone could help us hone in on the correct way, and then I can post the output from that. I suspect that the problem may have something to do with the config for Kafka using a hostname of 'sandbox.hortonworks.com' but i'm not sure. Any help would be GREATLY appreciated, we've really been banging our heads against the wall. We must be missing something. Many thanks, T
... View more
Labels:
- Labels:
-
Apache Kafka