question Re: sentry + hive + kerberos resource management in Archives of Support Questions (Read Only)

sentry + hive + kerberos resource management

ben123 — Tue, 21 Apr 2026 13:58:18 GMT

I have enabled Sentry to work with HiveServer2 with Kerberos Authentication. Therefore, impersonication on HiveServer2 is turned off.

Now all queries are run as 'hive' from Hue Hive UI, and oozie hive action.

How does resource management (YARN resource pool) works in this case? I want jobs to go into right pool, but now all Hive jobs are going into root.hive pool.

Samething happens with Impala when using llma. All impala jobs goes into root.llama pool.

Thank you

Ben

Re: sentry + hive + kerberos resource management

Darren — Fri, 26 Jun 2015 23:46:23 GMT

Hi Ben,

The hive+sentry issue sounds like an issue that was fixed in CDH5.2.1 and CDH5.3.0+. What version of CDH are you using? Are you seeing the same problem for jobs launched from command-line (as a user other than hive, of course), or only ones launched through hue and oozie?

I'm not sure if we had any releases where llama was known to have this issue.

Thanks,
Darren

Re: sentry + hive + kerberos resource management

ben123 — Mon, 06 Jul 2015 13:17:52 GMT

sorry for late response Darren,

I'm using CDH 5.4.1.

This doesn't happen from command-line. If I'm authenticated as ben on shell environment, then the job gets submitted as ben.

On hue+oozie environment, if I submit a workflow job, oozie job-launcher get's submitted as the authenticated user ben. However actual hive job gets submitted as hive user.

Thank you.

Ben

Re: sentry + hive + kerberos resource management

ben123 — Mon, 06 Jul 2015 13:38:24 GMT

what's the issue tracking URL on 5.2.1 release? can't find it on Google 😞

Re: sentry + hive + kerberos resource management

Darren — Mon, 06 Jul 2015 17:45:28 GMT

When you ran from command-line, did you use "hive" or "beeline"? I forgot to clarify that you should test using the "beeline" client so it goes through HS2 and fully integrates with Sentry. This is also more similar to how Hue works (it talks to HS2).

When using Sentry, you are supposed to disable impersonation for HS2, which means that all jobs will be submitted as user "hive". When looking up permissions in Sentry and/or deciding which YARN pool to run in, however, it should use the submitting user, not "hive". So it isn't necessarily a problem that the hive job is submitted as the hive user.

Thanks,
Darren

Re: sentry + hive + kerberos resource management

ben123 — Mon, 06 Jul 2015 23:04:36 GMT

I tested on both hive and beeline, and running from command line works as intented. jobs get assigned to correct user/group queues.

Can you explain why it's ok for hive jobs get submitted as 'hive' user?

We have four different teams using Cloudera, and it gets difficult to manage resources if all hive jobs go to "root.hive" queue. And since "root.hive" queue has limited resouces allocated, most hive jobs will fail.

This is our job history.

application_1436195699910_0031	hive	INSERT INTO TABLE ...(Stage-1)	MAPREDUCE	root.hive	Mon Jul 6 15:44:38 -0500 2015	Mon Jul 6 15:45:11 -0500 2015	FINISHED	SUCCEEDED
application_1436195699910_0030	ben	oozie:launcher:T=hive2:W=JobName:A=hive2-6df2:ID=0000004-150706101622653-oozie-oozi-W	MAPREDUCE	root.infra	Mon Jul 6 15:44:22 -0500 2015	Mon Jul 6 15:45:21 -0500 2015	FINISHED	SUCCEEDED

other workflow actions such as sqoop/pig run on correct user/group queue.

I think this is problem with our cluster configuration, but please guide us with right direction 🙂

thank you for your help

Ben

Re: sentry + hive + kerberos resource management

Wilfred — Tue, 07 Jul 2015 07:27:58 GMT

The fact that the job runs as the hive user is correct. You have impersonation turned off when you turned on Sentry, at least that is what you should have done. The Hive user is thus the user that executes the job.

However the end user should be used to retrieve which queue the application is submitted in (if you use the FairScheduler). This does require some configuration on your side to make this work. There is a Knowledge Base article in our support portal on how to set that up for CM and non CM clusters. Search for "Hive FairScheduler".

I can remember already providing the steps using CM before on the forum:

Login to Cloudera Manager
Navigate to Cluster > Yarn > Instances > ResourceManager > Processes
Click on the link fair-scheduler.xml, this will open a new tab or window
Copy the contents into the a new file called: fair-scheduler.xml
On the HiveServer2 host create a new directory to store the xml file (for example, /etc/hive/fsxml)
Note: This file should not be placed in the standard Hive configuration directory since that directory is managed by Cloudera Manager and the file could be removed when changing other configuration settings.
Upload the fair-scheduler.xml file to the above created directory
In Cloudera Manager navigate to Cluster > Hive > Service-Wide > Advanced > Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml and add the following property:
```
<property>
  <name>yarn.scheduler.fair.allocation.file</name>
  <value>/etc/hive/fsxml/fair-scheduler.xml</value>
</property>
```
Save changes
Restart the Hive Service

NOTE: you must have the follwoing rule as the first rule in the placement policy:

<rule name="specified" />

Wiflred

Re: sentry + hive + kerberos resource management

ben123 — Thu, 09 Jul 2015 14:46:33 GMT

Tara!

Thank very much for your help. Now I understand that the job runs as hive user but the job will go to the designated queue. And after following your steps it worked 🙂

Initially I changed Placement Rules on resource pools and did not have "specified" pool as first rule.

Do I need to replace the local /etc/hive/fsxml/fair-scheduler.xml everytime I make changes to the "Dynamic Resource Pools"? I'm using CM cluster.

Best,

Ben

Re: sentry + hive + kerberos resource management

Darren — Thu, 09 Jul 2015 18:08:35 GMT

Until this bug is fixed, yes, you'll need to replace /etc/hive/fsxml/fair-scheduler.xml every time you change Yarn's copy of fair-scheduler.xml.

Thanks,
Darren

Re: sentry + hive + kerberos resource management

Madhu Ayanala — Wed, 09 Dec 2015 16:44:35 GMT

Hi, we have a similar issue and wondering if those steps listed are the resolution.

we have our cluster kerberised and we also deployed Sentry, as part of the setup in hive we disabled impersonation. so all the HIVE queries are being executed by the HIVE user.
We configured Dynamic resource manager pools, setting up 3 queues. HighPriority, LowPriority and Default.
Everybody can submit jobs to the default queue, that is working as expected.
The HighPriority, LowPriority are managed by group membership to two different AD groups.

I assigned a test user both groups so it could submit jobs to both queues (HighPriority, LowPriority) when i submitted a job
we got the following error message

ERROR : Job Submission failed with exception 'java.io.IOException(Failed to run job : User hive cannot submit applications to queue root.HighPriority)'
java.io.IOException: Failed to run job : User hive cannot submit applications to queue root.HighPriority

this is correct because the hive user doesn't is not a member of any of those groups.
I modified the submission access control to add the hive user to the pool and this time the job completed, however that breaks the access control model we are trying to implement because now all hive users can make use of both pools even though they don't belong any of the AD groups that are supposed to be controlling who can submit jobs to the pool.

Is there a way to control which users can submit to specific resource pools in HIVE and leverage the Ad groups created for this purpose?

Re: sentry + hive + kerberos resource management

ywheel — Fri, 08 Dec 2017 10:27:00 GMT

Hi, we have the similar issue. I use CDH 5.6.0. Is this bug fixed? However, I have no idea about the right way to make it work.

I prefer to use the Placement Rules configurated in YARN instead of this workaround method.

And our situation is quit similar with Madhu's. I have setup 3 queues, and there are 3 groups in LDAP(these 3 groups are also in Linux OS). Is there a way to control which users can submit to specific resource pools in HIVE?

Thanks,

ywheel

Re: sentry + hive + kerberos resource management

Wilfred — Fri, 08 Dec 2017 11:35:53 GMT

This has been fixed in later releases of Cloudera Manager and CDH. When you manage the cluster through CM the config, and changes later on, will be automatically deployed to hive server.

Also don't forget to make sure that hive user must have permission to submit to all queues. Simplest way is to add Hive to the root submit queue ACL.

Wilfred

Re: sentry + hive + kerberos resource management

ywheel — Fri, 08 Dec 2017 11:48:03 GMT

Thanks a lot!

Since which version of CDH and Cloudera Manager have this fixed feature? I'm on CHD 5.6.0, and seems it doesn't work yet.

Re: sentry + hive + kerberos resource management

Wilfred — Fri, 08 Dec 2017 11:50:28 GMT

You must be on 5.8.0 or later for both CDH and CM

Wilfred

Re: sentry + hive + kerberos resource management

ywheel — Fri, 08 Dec 2017 11:53:35 GMT

Got it! Thanks for your quick reply.

Best,
ywheel

Re: sentry + hive + kerberos resource management

huadao — Wed, 20 Dec 2017 01:46:00 GMT

Hi, we have a similar issue with Madhu's, by the way, We are using CDH 5.12.0

The following is Madhu's describe:

I assigned a test user both groups so it could submit jobs to both queues (HighPriority, LowPriority) when i submitted a job
we got the following error message

ERROR : Job Submission failed with exception 'java.io.IOException(Failed to run job : User hive cannot submit applications to queue root.HighPriority)'
java.io.IOException: Failed to run job : User hive cannot submit applications to queue root.HighPriority

this is correct because the hive user doesn't is not a member of any of those groups.
I modified the submission access control to add the hive user to the pool and this time the job completed, however that breaks the access control model we are trying to implement because now all hive users can make use of both pools even though they don't belong any of the AD groups that are supposed to be controlling who can submit jobs to the pool.

Is there a way to control which users can submit to specific resource pools in HIVE and leverage the Ad groups created for this purpose?

Re: sentry + hive + kerberos resource management

Wilfred — Wed, 20 Dec 2017 07:29:35 GMT

The placement rules are executed as the original user. That means the job will be added to the correct pool. The end user can not override that because the mapred.job.queuename property should be blacklisted.

The hive user should never be accessible for any user, it is a service principal and allowing it to be used by end users will give you far bigger issues.

I thus do not see how adding hive as a user to the acl breaks it.

Wilfred

Re: sentry + hive + kerberos resource management

SachaH — Wed, 20 Dec 2017 15:26:19 GMT

Hi Wilfried,

I'm sorry to ask again, but i'm facing the same problem and I don't understand how to configure Dynamic Ressource Pool Configuration to work using orginal user groups (me not hive).

I'm using CDH 5.13 with Kerberos and Sentry. As I am using Sentry, impersonation is disabled.

My configuration is

root

|--A

|--B

On root, submission ACL are set to allow only "sentry" user to submit in this pool

On A, submission ACL are set to allow only group A to submit in this pool

On B, submission ACL are set to allow only group B to submit in this pool

Placement rules are :

1 - "Use the pool Specified at run time, only if the pool exists."

2 - "Use the pool root.[username] and create the pool if it does not exist. "

When I submit a query with a user from the group A, using Hue and setting "set mapred.job.queue.name=A;" I got the error : "User hive cannot submit applications to queue root.A"

If I add hive to allowed user on root, the query is working fine but both A and B user's can submit query

If I add hive to only "A" resource pool, then user from A and B group can submit query to ressource pool A, but none can submit to resource pool B

Maybe I am missing an important part, but I don't have the same behavior as you explained and if I add hive in authorized user it will break the ACL's as every user could use all the resource pool.

Can you give us the good configuration to have the same behavior as your's ?