Hello ManuelCalvo   I don't know if it's asking you too much, as I would have to do a relationship of publisher and consumer. I'm a little new to the subject of Big data. Thank you very much for the info.   Greetings ... View more

@ManuelCalvo  [root@w01 bin]# ./kafka-console-producer.sh --broker-list w01.s03.hortonweb.com:6667 --topic PruebaNYC --producer-property security.protocol=SASL_PLAINTEXT >Hello >Helo     Thank you very much it worked for me, I don't know if it is to ask a lot and what would be the command for the consumer. Thank you ... View more

Hello @ManuelCalvo      That's right. We're using Kerberos. Generate a ticket as admin and execute the command indicated and consulted a Broker. I attach a URL with the report that comes out.   https://ibb.co/Bg1jfQx   I also indicate the commands that we executed when we created the topic and the producer.   ./kafka-topics.sh --create --zookeeper m01.s02.hortonweb.com:2181 --replication-factor 3 --partitions 1 --topic PruebaKafka (I Have 3 Brokers) Created topic "PruebaKafka". ./kafka-console-producer.sh --broker-list m01.s02.hortonweb.com:6667 --topic PruebaKafka >Prueba mensaje [2019-09-26 12:22:22,059] WARN [Producer clientId=console-producer] Bootstrap broker w01.s03.hortonweb.com.com:6667 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient) Thank you ... View more

Hello @ManuelCalvo    Generate a ticket with the kerberos admin and execute the command. But I don't see any error... excuse me I'm a little new to this topic. I attach the result of the command.   [root@m01 bin]# ./zkCli.sh -server m01.s02.hortonweb.com:2181 get /brokers/ids/1001 Connecting to m01.s02.hortonweb.com:2181 2019-09-26 12:09:27,940 - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-78--1, built on 12/06/2018 12:30 GMT 2019-09-26 12:09:27,942 - INFO [main:Environment@100] - Client environment:host.name=m01.s02.hortonweb.com 2019-09-26 12:09:27,942 - INFO [main:Environment@100] - Client environment:java.version=1.8.0_112 2019-09-26 12:09:27,944 - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-09-26 12:09:27,944 - INFO [main:Environment@100] - Client environment:java.home=/usr/jdk64/jdk1.8.0_112/jre 2019-09-26 12:09:27,944 - INFO [main:Environment@100] - Client environment:java.class.path=/usr/hdp/current/zookeeper-client/bin/../build/classes:/usr/hdp/current/zookeeper-client/bin/../build/lib/*.jar:/usr/hdp/current/zookeeper-client/bin/../lib/slf4j-log4j12-1.6.1.jar:/u sr/hdp/current/zookeeper-client/bin/../lib/slf4j-api-1.6.1.jar:/usr/hdp/current/zookeeper-client/bin/../lib/netty-3.10.5.Final.jar:/usr/hdp/current/zookeeper-client/bin/../lib/log4j-1.2.16.jar:/usr/hdp/current/zookeeper-client/bin/../lib/jline-0.9.94.jar:/usr/hdp/current/zoo keeper-client/bin/../zookeeper-3.4.6.3.1.0.0-78.jar:/usr/hdp/current/zookeeper-client/bin/../src/java/lib/*.jar:/usr/hdp/current/zookeeper-client/bin/../conf::/usr/share/zookeeper/* 2019-09-26 12:09:27,944 - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-09-26 12:09:27,944 - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-09-26 12:09:27,945 - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-09-26 12:09:27,945 - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-09-26 12:09:27,945 - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-09-26 12:09:27,945 - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.12.1.el7.x86_64 2019-09-26 12:09:27,945 - INFO [main:Environment@100] - Client environment:user.name=root 2019-09-26 12:09:27,945 - INFO [main:Environment@100] - Client environment:user.home=/root 2019-09-26 12:09:27,945 - INFO [main:Environment@100] - Client environment:user.dir=/usr/hdp/3.1.0.0-78/zookeeper/bin 2019-09-26 12:09:27,947 - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=m01.s02.hortonweb.com:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@67424e82 2019-09-26 12:09:28,051 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):Login@294] - successfully logged in. 2019-09-26 12:09:28,052 - INFO [Thread-0:Login$1@127] - TGT refresh thread started. 2019-09-26 12:09:28,056 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ZooKeeperSaslClient$1@289] - Client will use GSSAPI as SASL mechanism. 2019-09-26 12:09:28,067 - INFO [Thread-0:Login@302] - TGT valid starting at: Thu Sep 26 09:16:58 CEST 2019 2019-09-26 12:09:28,067 - INFO [Thread-0:Login@303] - TGT expires: Thu Sep 26 19:16:58 CEST 2019 2019-09-26 12:09:28,067 - INFO [Thread-0:Login$1@181] - TGT refresh sleeping until: Thu Sep 26 17:26:26 CEST 2019 2019-09-26 12:09:28,104 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1019] - Opening socket connection to server m01.s02.hortonweb.com/192.168.0.2:2181. Will attempt to SASL-authenticate using Login Context section 'Client' 2019-09-26 12:09:28,160 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@864] - Socket connection established, initiating session, client: /192.168.0.2:59854, server: m01.s02.hortonweb.com/192.168.0.2:2181 2019-09-26 12:09:28,317 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1279] - Session establishment complete on server m01.s02.hortonweb.com/192.168.0.2:2181, sessionid = 0x16ccd8510b02493, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null WATCHER:: WatchedEvent state:SaslAuthenticated type:None path:null {"listener_security_protocol_map":{"SASL_PLAINTEXT":"SASL_PLAINTEXT"},"endpoints":["SASL_PLAINTEXT://w01.s03.hortonweb.com:6667"],"jmx_port":-1,"host":null,"timestamp":"1569423123514","port":-1,"version":4} cZxid = 0x6c420 ctime = Wed Sep 25 16:52:03 CEST 2019 mZxid = 0x6c420 mtime = Wed Sep 25 16:52:03 CEST 2019 pZxid = 0x6c420 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x16ccd8510b0238e dataLength = 205 numChildren = 0 [root@m01 bin]#   I execute these commands ./kafka-topics.sh --create --zookeeper m01.s02.hortonweb.com:2181 --replication-factor 3 (i have 3 Brokers)--partitions 1 --topic PruebaKafka Created topic "PruebaKafka". ./kafka-console-producer.sh --broker-list w01.s03.hortonweb.com:6667 --topic PruebaKafka >Helpme [2019-09-26 12:22:22,059] WARN [Producer clientId=console-producer] Bootstrap broker w01.s03.hortonweb.com.com:6667 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient) [2019-09-26 12:22:22,165] WARN [Producer clientId=console-producer] Bootstrap broker w01.s03.hortonweb.com.com:6667 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient) [2019-09-26 12:22:22,270] WARN [Producer clientId=console-producer] Bootstrap broker w01.s03.hortonweb.com.com:6667 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient) Greetings ... View more

Hello @ManuelCalvo    When you execute the command [zookeeper@m01 bin]$ ./zkCli.sh -server m01.s02.hortonweb.com:2181 get /brokers/ids Connecting to m01.s02.hortonweb.com:2181 2019-09-25 16:22:54,331 - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-78--1, built on 12/06/2018 12:30 GMT 2019-09-25 16:22:54,333 - INFO [main:Environment@100] - Client environment:host.name=m01.s02.hortonweb.com 2019-09-25 16:22:54,333 - INFO [main:Environment@100] - Client environment:java.version=1.8.0_112 2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.home=/usr/jdk64/jdk1.8.0_112/jre 2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.class.path=/usr/hdp/current/zookeeper-client/bin/../build/classes:/usr/hdp/current/zookeeper-client/bin/../build/lib/*.jar:/usr/hdp/current/zookeeper-client/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/hdp/ current/zookeeper-client/bin/../lib/slf4j-api-1.6.1.jar:/usr/hdp/current/zookeeper-client/bin/../lib/netty-3.10.5.Final.jar:/usr/hdp/current/zookeeper-client/bin/../lib/log4j-1.2.16.jar:/usr/hdp/current/zookeeper-client/bin/../lib/jline-0.9.94.jar:/usr/hdp/current/zookeeper-client/ bin/../zookeeper-3.4.6.3.1.0.0-78.jar:/usr/hdp/current/zookeeper-client/bin/../src/java/lib/*.jar:/usr/hdp/current/zookeeper-client/bin/../conf::/usr/share/zookeeper/* 2019-09-25 16:22:54,335 - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.12.1.el7.x86_64 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:user.name=zookeeper 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:user.home=/home/zookeeper 2019-09-25 16:22:54,336 - INFO [main:Environment@100] - Client environment:user.dir=/usr/hdp/3.1.0.0-78/zookeeper/bin 2019-09-25 16:22:54,337 - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=m01.s02.hortonweb.com:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@67424e82 2019-09-25 16:22:54,367 - WARN [main-SendThread(m01.s02.hortonweb.com:2181):ZooKeeperSaslClient$ClientCallbackHandler@496] - Could not login: the client is being asked for a password, but the Zookeeper client code does not currently support obtaining a password from the user. Make sure that the client is configured to use a ticket cache (using the JAAS configuration setting 'useTicketCache=true)' and restart the client. If you still get this message after that, the TGT in the ticket cache has expired and must be manually refreshed. To do so, first determine if you are using a password or a keytab. If the former, run kinit in a Unix shell in the environment of the user who is running this Zookeeper client using the command 'kinit <princ>' (where <princ> is the name of the client's Kerberos principal). If the latter, do 'kinit -k -t <ke ytab> <princ>' (where <princ> is the name of the Kerberos principal, and <keytab> is the location of the keytab file). After manually refreshing your cache, restart this client. If you continue to see this message after manually refreshing your cache, ensure that your KDC host's cl ock is in sync with this host's clock. 2019-09-25 16:22:54,369 - WARN [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1001] - SASL configuration failed: javax.security.auth.login.LoginException: No password provided Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2019-09-25 16:22:54,371 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1019] - Opening socket connection to server m01.s02.hortonweb.com/192.168.0.2:2181 WATCHER:: WatchedEvent state:AuthFailed type:None path:null 2019-09-25 16:22:54,443 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@864] - Socket connection established, initiating session, client: /192.168.0.2:57438, server: m01.s02.hortonweb.com/192.168.0.2:2181 2019-09-25 16:22:54,571 - INFO [main-SendThread(m01.s02.hortonweb.com:2181):ClientCnxn$SendThread@1279] - Session establishment complete on server m01.s02.hortonweb.com/192.168.0.2:2181, sessionid = 0x16ccd8510b02378, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null null cZxid = 0x239 ctime = Tue May 07 10:07:46 CEST 2019 mZxid = 0x239 mtime = Tue May 07 10:07:46 CEST 2019 pZxid = 0x6b559 cversion = 103 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 3   I'm a little new to this subject. But I am impressed that by having 2 network cards my requests are going off without a specific direction. I have one for my LAN and one for WAN. both in Broker and Zookeeper. Greetings ... View more