Support Questions
Find answers, ask questions, and share your expertise

Kerberos Logging

Rising Star

Hello,
How do I check the location where Kerberos is writing logs?
I checked the location which is mentioned in krb5.conf (default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log) however the log files mentioned in this location are empty. Am I checking an incorrect location?

 

Secondly logs on Hue Server & KT renewer is not getting updated / current.

 

Any help/guidance is appreciated.

1 ACCEPTED SOLUTION

Master Collaborator

@wert_1311 ,

 

Kerberos issue can be a hard thing to troubleshoot. I've done a LOT of troubleshooting involving Kerberos and I still find myself stumped at times.

 

What's in the krb5kdc.log rarely matters. The best way to find out what's going on is to look at the client log.

Enable debug logging for your application and ensure you also toggle debug mode for the Kerberos modules with -Dsun.security.krb5.debug=true for your application.

 

Look at and read the logs carefully, and google a lot 🙂 Kerberos debug logs are rarely intuitive.

If you can share some of that here it might help us find some clues as to what is going on. If you can't, I understand.

 

Good luck.

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

View solution in original post

15 REPLIES 15

Master Collaborator

Hi @wert_1311 ,

 


How do I check the location where Kerberos is writing logs?
I checked the location which is mentioned in krb5.conf (default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log) however the log files mentioned in this location are empty. Am I checking an incorrect location?

Can you explain what you are trying to verify/check?

What sort of Kerberos information are you looking for? What kind of problem are you troubleshooting?

 

Secondly logs on Hue Server & KT renewer is not getting updated / current.

Do you mean that the logs have not been updated for a while? Is the disk full?

What's the location of the logs that you're looking at?

Have you double-checked that you are checking the logs in the correct hosts, where the Hue Server and KT Renewer is deployed to?

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Rising Star

Hi @araujo 

Appericate your reply, regarding the first part of the question, I am trying to check the KDC server logs to troubleshoot an issue wherein an application is unable to renew its ticket, hence wanted to check the logs.
Now for the second issue, the logs are not getting written in /var/log/hue/ (on the host where the roles are configured) for kt_renewer.log & runcpserver.log as the timestamp on both on them show Nov 24. There is ample Disk Space available.

 

Thanks 

Wert

Master Collaborator

@wert_1311 ,

 

Appericate your reply, regarding the first part of the question, I am trying to check the KDC server logs to troubleshoot an issue wherein an application is unable to renew its ticket, hence wanted to check the logs.

Instead of the krb5.conf, please check the log location in the kdc.conf file. Assuming you're using a MIT KDC, this file is typically found at /var/kerberos/krb5kdc/kdc.conf on the KDC server


Now for the second issue, the logs are not getting written in /var/log/hue/ (on the host where the roles are configured) for kt_renewer.log & runcpserver.log as the timestamp on both on them show Nov 24. There is ample Disk Space available.

This is a bit strange. Either the service is in a funny state or the location of the logs were changed. Check the service configuration and look for the latest configured location. Also ensure you are looking at the right server, where the roles should be running.

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Rising Star

Hi @araujo,

Yes we are using MIT Kerberose, below is the configs of /var/kerberos/krb5kdc/kdc.conf

 

Spoiler

[kdcdefaults]
kdc_ports = 88
kdc_tcp_ports = 88

[realms]
MY_COMPANY.COM = {
#master_key_type = aes256-cts
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
max_renewable_life = 7d 0h 0m 0s
forwardable = true
udp_preference_limit = 1
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 default_principal_flags = +renewable, +forwardable
database_name = /opt/localkrb5/krb5kdc/principal
}

I do not see any infomation about logging in this.

 

 

Regarding the second issue I have found a solution - https://my.cloudera.com/knowledge/Logs-are-not-updating-in-varloghue-after-upgrading-to-CDH-6?id=878... 

 

Thanks 

Wert

Master Collaborator

Good to hear! Which CDH/CDP version are you using?

 

Do you have any application error message regarding the failure of renewing the ticket?

 

André

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Rising Star

CM/ CDH 6.3.3

Currently do not the error screenshot etc.

Any infomation on where the logs would be?

 

Thanks

Wert

 

Master Collaborator

Let's start from the beginning 🙂

What are the symptoms you see?

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Master Collaborator

@wert_1311 ,

 

What are the symptoms of the ticket renewal failure? Are there any error messages anywhere?

How did you conclude it's a ticket renewal problem?

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Rising Star

Hi @araujo 

 

What are the symptoms of the ticket renewal failure? Are there any error messages anywhere?

> We see GSSException on Application logs

 

How did you conclude it's a ticket renewal problem?

> We haven’t concluded its a renewal problem however to rule out kerberose issue, we need logs which at the present is not getting written in the location specified in krb5.conf file.

 

Thanks 

Wert

Master Collaborator

@wert_1311 ,

 

Would you be able to share the stack traces from the application logs?

 

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Rising Star

@araujo 

 

Cant do that for securtiy / policy reasons, hope you understand.

 

Thanks 

Wert

Master Collaborator

@wert_1311 ,

 

Totally understandable.

What's your OS?

 

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Master Collaborator

@wert_1311 ,

 

Kerberos issue can be a hard thing to troubleshoot. I've done a LOT of troubleshooting involving Kerberos and I still find myself stumped at times.

 

What's in the krb5kdc.log rarely matters. The best way to find out what's going on is to look at the client log.

Enable debug logging for your application and ensure you also toggle debug mode for the Kerberos modules with -Dsun.security.krb5.debug=true for your application.

 

Look at and read the logs carefully, and google a lot 🙂 Kerberos debug logs are rarely intuitive.

If you can share some of that here it might help us find some clues as to what is going on. If you can't, I understand.

 

Good luck.

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Rising Star

Hi @araujo 

 

Appericate all your assistance 

 

Thanks

Wert

Master Collaborator

@wert_1311 ,

 

Any luck with this? Found the root cause?

 

André

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.
; ;