Hive metastore and postgres authentication to Kerberos infrastructure

hduraiswamy — Tue, 28 Feb 2017 13:32:39 GMT

I am working with a customer who complains of recurring production issue (about once a month) due to overloading auth requests to their Kerberos infrastructure (10’s of thousands of auth attempts within a very short time-frame) - and any help with the below questions would be very appreciated.

Apparently, these requests come from their Hive Metastore Service (aka HMS) account “hcatalog” and their postgres database host. The customer would like better understand how HMS and the postgres metastore handle authentication requests ??

It kind of makes sense to have some form of ticket caching to keep these auth attempts fairly low – no?
- If yes, that should be the expectation. Is this driven by some kind configuration on the HMS or Postgre side (that the customer has perhaps, either mis-configured or missing) ?

Thanks and let me know your thoughts.

Re: Hive metastore and postgres authentication to Kerberos infrastructure

hduraiswamy — Wed, 08 Mar 2017 03:20:27 GMT

Working further with our support team and customer, it was determined that this issue was coming mostly from Postgres side. The reason being the ticket caching was not enabled for the PG side, and the customer is currently working on enabling the same.

This document link should talk about enabling caching from Postgres - http://jpmens.net/2012/06/23/postgresql-and-kerberos/

As far as the above question on multiple requests on the same session goes - Yes, Hive metastore does caching my default, and the multiple commands executed within the same HS2 session is translated to a single auth request due to caching at the HMS level

question Re: Hive metastore and postgres authentication to Kerberos infrastructure in Archives of Support Questions (Read Only)

Hive metastore and postgres authentication to Kerberos infrastructure

Re: Hive metastore and postgres authentication to Kerberos infrastructure