Gold ⭐️

Solved

Error Kafka

8 replies
24311 views
Translate All

Hey, folks,

I have an error when executing kafka when executing a message for the topic.

kafka-console-producer.sh --broker-list w01.s03.hortonweb.com:6667 -topic TestNYC

error.

WARN [Producer clientId=console-producer] Bootstrap broker w01.s03.hortonweb.com:6667 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

I have 3 Brokers, which are working and is configured according to the parameters. just a topic that I just realized.

I have 2 network cards one internal and external to netstat I see that port 6667 is listening to the internal. but still executing the command with the internal IP

kafka-console-producer.sh --broker-list 192.168.0.9:6667 -topic TestNYC

I still get the same mistake.

Someone could help me.

Greetings

Apache Kafka

ManuelCalvoAnswer

Gold ⭐️

@Peruvian81

You can try below command for the consumer:

./kafka-console-consumer.sh --bootstrap-server w01.s03.hortonweb.com:6667 --topic PruebaNYC --consumer-property security.protocol=SASL_PLAINTEXT --from-beginning

If that solves your issue, kindly put this thread as solved.

Thanks.

ManuelCalvo

Gold ⭐️

@Peruvian81

Hi,

If you connect to the zookeeper cli using:

ZK_HOME/zookeeper-client/bin/zkCli.sh -server <zkHost>:<zkPort>

Then you can run: get /brokers/ids/<brokerID> and check in the "endpoints" where the kafka brokers are listening.

Then try using that security-protocol ip:port to connect to the brokers.

Also, make sure that the topic has all their replicas in sync by running "describe" command line.

Let us know how it goes.

Translate

Peruvian81Author

Gold ⭐️

Hello @ManuelCalvo

When you execute the command

[zookeeper@m01 bin]$ ./zkCli.sh -server m01.s02.hortonweb.com:2181 get /brokers/ids
Connecting to m01.s02.hortonweb.com:2181
2019-09-25 16:22:54,331 - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-78--1, built on 12/06/2018 12:30 GMT
2019-09-25 16:22:54,333 - INFO [main:Environment@100] - Client environment:host.name=m01.s02.hortonweb.com
2019-09-25 16:22:54,333 - INFO [main:Environment@100] - Client environment:java.version=1.8.0_112
2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.home=/usr/jdk64/jdk1.8.0_112/jre
2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.class.path=/usr/hdp/current/zookeeper-client/bin/../build/classes:/usr/hdp/current/zookeeper-client/bin/../build/lib/*.jar:/usr/hdp/current/zookeeper-client/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/hdp/
current/zookeeper-client/bin/../lib/slf4j-api-1.6.1.jar:/usr/hdp/current/zookeeper-client/bin/../lib/netty-3.10.5.Final.jar:/usr/hdp/current/zookeeper-client/bin/../lib/log4j-1.2.16.jar:/usr/hdp/current/zookeeper-client/bin/../lib/jline-0.9.94.jar:/usr/hdp/current/zookeeper-client/
bin/../zookeeper-3.4.6.3.1.0.0-78.jar:/usr/hdp/current/zookeeper-client/bin/../src/java/lib/*.jar:/usr/hdp/current/zookeeper-client/bin/../conf::/usr/share/zookeeper/*
2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.12.1.el7.x86_64
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:user.name=zookeeper
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:user.home=/home/zookeeper
2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:user.dir=/usr/hdp/3.1.0.0-78/zookeeper/bin
2019-09-25 16:22:54,337 - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=m01.s02.hortonweb.com:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@67424e82
2019-09-25 16:22:54,367 - WARN [main-SendThread(m01.s02.hortonweb.com:2181):ZooKeeperSaslClient$ClientCallbackHandler@496] - Could not login: the client is being asked for a password, but the Zookeeper client code does not currently support obtaining a password from the user. Make
sure that the client is configured to use a ticket cache (using the JAAS configuration setting 'useTicketCache=true)' and restart the client. If you still get this message after that, the TGT in the ticket cache has expired and must be manually refreshed. To do so, first determine
if you are using a password or a keytab. If the former, run kinit in a Unix shell in the environment of the user who is running this Zookeeper client using the command 'kinit <princ>' (where <princ> is the name of the client's Kerberos principal). If the latter, do 'kinit -k -t <ke
ytab> <princ>' (where <princ> is the name of the Kerberos principal, and <keytab> is the location of the keytab file). After manually refreshing your cache, restart this client. If you continue to see this message after manually refreshing your cache, ensure that your KDC host's cl
ock is in sync with this host's clock.
2019-09-25 16:22:54,369 - WARN [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1001] - SASL configuration failed: javax.security.auth.login.LoginException: No password provided Will continue connection to Zookeeper server without SASL authentication, if Zookeeper
server allows it.
2019-09-25 16:22:54,371 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1019] - Opening socket connection to server m01.s02.hortonweb.com/192.168.0.2:2181

WATCHER::

WatchedEvent state:AuthFailed type:None path:null
2019-09-25 16:22:54,443 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@864] - Socket connection established, initiating session, client: /192.168.0.2:57438, server: m01.s02.hortonweb.com/192.168.0.2:2181
2019-09-25 16:22:54,571 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1279] - Session establishment complete on server m01.s02.hortonweb.com/192.168.0.2:2181, sessionid = 0x16ccd8510b02378, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
null
cZxid = 0x239
ctime = Tue May 07 10:07:46 CEST 2019
mZxid = 0x239
mtime = Tue May 07 10:07:46 CEST 2019
pZxid = 0x6b559
cversion = 103
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 3

I'm a little new to this subject. But I am impressed that by having 2 network cards my requests are going off without a specific direction. I have one for my LAN and one for WAN. both in Broker and Zookeeper.

Greetings

Translate

ManuelCalvo

Gold ⭐️

@Peruvian81

Are you using kerberos? If yes, make sure you have a valid ticket in order to avoid below exception:

2019-09-25 16:22:54,367 - WARN [main-SendThread(m01.s02.hortonweb.com:2181):ZooKeeperSaslClient$ClientCallbackHandler@496] - Could not login: the client is being asked for a password, but the Zookeeper client code does not currently support obtaining a password from the user. Make
sure that the client is configured to use a ticket cache (using the JAAS configuration setting 'useTicketCache=true)' and restart the client. If you still get this message after that, the TGT in the ticket cache has expired and must be manually refreshed. To do so, first determine
if you are using a password or a keytab. If the former, run kinit in a Unix shell in the environment of the user who is running this Zookeeper client using the command 'kinit <princ>' (where <princ> is the name of the client's Kerberos principal). If the latter, do 'kinit -k -t <ke
ytab> <princ>' (where <princ> is the name of the Kerberos principal, and <keytab> is the location of the keytab file). After manually refreshing your cache, restart this client. If you continue to see this message after manually refreshing your cache, ensure that your KDC host's cl
ock is in sync with this host's clock.

From the command line, please add the broker id: get /brokers/ids/<brokerID>

Example:

ZK_HOME/zookeeper-client/bin/zkCli.sh -server host:2181 get /brokers/ids/1001

If you don't know your current ids, you can get them by using:

ZK_HOME/zookeeper-client/bin/zkCli.sh -server host:2181 ls /brokers/ids

Translate

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.