Unknown Question

[#1]

...

Question is no longer available.

[#2]

@Fahad Sarwar

After kerberization you now need to have a valid Kerberos ticket to launch any job.

To grab a a valid kerberos ticke see example below

###################################################3
# How to check the principals of a keytab
####################################################
[kafka@{host} ~]$ klist -ket /etc/security/keytabs/kafka.headless.keytab
Keytab name: FILE:/etc/security/keytabs/kafka.headless.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   1 12/28/16 22:25:31 kafka-{host}@REALM (des3-cbc-sha1)
   1 12/28/16 22:25:31 kafka-{host}@REALM (aes128-cts-hmac-sha1-96)
   1 12/28/16 22:25:31 kafka-{host}@REALM (arcfour-hmac)
   1 12/28/16 22:25:31 kafka-{host}@REALM (des-cbc-md5)
   1 12/28/16 22:25:31 kafka-{host}@REALM (aes256-cts-hmac-sha1-96)
##############################################
# From the above the prinicipal is kafka-{host}
################################################
[kafka@{host} ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_507
Default principal: kafka-{host}@REALM
Valid starting     Expires            Service principal
12/28/16 22:57:11  12/29/16 22:57:11  krbtgt/REALM@REALM renew until 12/28/16 22:57:11
12/28/16 22:57:11  12/29/16 22:57:11  HTTP/{host}.REALM@renew until 12/28/16 22:57:11
12/28/16 22:57:11  12/29/16 22:57:11  HTTP/{host}.nakuru.com@REALM renew until 12/28/16 22:57:11
################################
# Now run Kinit with the  correct principal
####################################
[kafka@{host} ~]$ kinit -kt /etc/security/keytabs/kafka.headless.keytab kafka-{host}
[kafka@{host} ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_507
Default principal: kafka-{host}@REALM
Valid starting     Expires            Service principal
12/28/16 23:42:15  12/29/16 23:42:15  krbtgt/@REALM renew until 12/28/16 23:42:15

Now with a valid ticket, you can launch your command

./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list ml-master3.mobilink.osa:6667 --topic test --time -1 --offsets 1 | awk -F ":" '{sum += $3} END {print Sum}'

[#3]

@Fahad Sarwar

You should grab a valid kerberos ticket, copy and paste the below code

# kinit -kt kafka.service.keytab kafka/ml-master3.mobilink.osa@MOBILINK.OSA

Then run your kafka code

# ./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list ml-master3.mobilink.osa:6667 --topic test --time -1 --offsets 1 | awk -F ":" '{sum += $3} END {print Sum}'

It should work let me know

[#4]

Unknown Answer

[#5]

Although kafka port is 6667 but when i execute below command it will pick message and give below error after a while:

./kafka-console-producer.sh --broker-list localhost:9092 --topic test

 ERROR Error when sending message to topic test with key: null, value: 6 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

[#6]

@Fahad Sarwar

Can you check your credentials?

Check contents of your kafka_client_jaas.conf

[#7]

@Fahad Sarwar

All the same your keytabs should have an entry in the kafka_client_jaas.conf for the kafka service authentication.

Can you copy and paste in here the contents of below files

kafka_client_jaas.conf
kafka_server_jaas.conf 

[#8]

@Fahad Sarwar @Laeeq Ahmad

Can you change the property in the server.properties file from

listeners=PLAINTEXT://hostname:{port} 

to

listeners=PLAINTEXT://0.0.0.0:{port}

Then restart the kafka process