Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unknown Question

Unknown Question

Explorer
...

Question is no longer available.
7 REPLIES 7
Highlighted

Re: Unknown Question

Mentor

@Fahad Sarwar

After kerberization you now need to have a valid Kerberos ticket to launch any job.

To grab a a valid kerberos ticke see example below

###################################################3
# How to check the principals of a keytab
####################################################
[kafka@{host} ~]$ klist -ket /etc/security/keytabs/kafka.headless.keytab
Keytab name: FILE:/etc/security/keytabs/kafka.headless.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   1 12/28/16 22:25:31 kafka-{host}@REALM (des3-cbc-sha1)
   1 12/28/16 22:25:31 kafka-{host}@REALM (aes128-cts-hmac-sha1-96)
   1 12/28/16 22:25:31 kafka-{host}@REALM (arcfour-hmac)
   1 12/28/16 22:25:31 kafka-{host}@REALM (des-cbc-md5)
   1 12/28/16 22:25:31 kafka-{host}@REALM (aes256-cts-hmac-sha1-96)
##############################################
# From the above the prinicipal is kafka-{host}
################################################
[kafka@{host} ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_507
Default principal: kafka-{host}@REALM
Valid starting     Expires            Service principal
12/28/16 22:57:11  12/29/16 22:57:11  krbtgt/REALM@REALM renew until 12/28/16 22:57:11
12/28/16 22:57:11  12/29/16 22:57:11  HTTP/{host}.REALM@renew until 12/28/16 22:57:11
12/28/16 22:57:11  12/29/16 22:57:11  HTTP/{host}.nakuru.com@REALM renew until 12/28/16 22:57:11
################################
# Now run Kinit with the  correct principal
####################################
[kafka@{host} ~]$ kinit -kt /etc/security/keytabs/kafka.headless.keytab kafka-{host}
[kafka@{host} ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_507
Default principal: kafka-{host}@REALM
Valid starting     Expires            Service principal
12/28/16 23:42:15  12/29/16 23:42:15  krbtgt/@REALM renew until 12/28/16 23:42:15

Now with a valid ticket, you can launch your command

./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list ml-master3.mobilink.osa:6667 --topic test --time -1 --offsets 1 | awk -F ":" '{sum += $3} END {print Sum}'
Highlighted

Re: Unknown Question

Mentor

@Fahad Sarwar

You should grab a valid kerberos ticket, copy and paste the below code

# kinit -kt kafka.service.keytab kafka/ml-master3.mobilink.osa@MOBILINK.OSA

Then run your kafka code

# ./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list ml-master3.mobilink.osa:6667 --topic test --time -1 --offsets 1 | awk -F ":" '{sum += $3} END {print Sum}'

It should work let me know

Highlighted

Re: Unknown Question

Explorer

Unknown Answer

Highlighted

Re: Unknown Question

Explorer

Although kafka port is 6667 but when i execute below command it will pick message and give below error after a while:

./kafka-console-producer.sh --broker-list localhost:9092 --topic test
 ERROR Error when sending message to topic test with key: null, value: 6 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
Highlighted

Re: Unknown Question

Mentor

@Fahad Sarwar

Can you check your credentials?

Check contents of your kafka_client_jaas.conf

Highlighted

Re: Unknown Question

Mentor

@Fahad Sarwar

All the same your keytabs should have an entry in the kafka_client_jaas.conf for the kafka service authentication.

Can you copy and paste in here the contents of below files

kafka_client_jaas.conf
kafka_server_jaas.conf 
Highlighted

Re: Unknown Question

Mentor

@Fahad Sarwar @Laeeq Ahmad

Can you change the property in the server.properties file from

listeners=PLAINTEXT://hostname:{port} 

to

listeners=PLAINTEXT://0.0.0.0:{port}

Then restart the kafka process

Don't have an account?
Coming from Hortonworks? Activate your account here