Member since
02-04-2016
132
Posts
52
Kudos Received
7
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
6195 | 07-25-2018 10:53 AM | |
1754 | 07-25-2018 05:15 AM | |
1816 | 10-03-2017 12:08 PM | |
3071 | 04-04-2017 05:36 AM | |
3323 | 11-29-2016 05:40 PM |
10-05-2016
07:55 AM
Hi Configuring SSSD OS level did the trick as per your suggestions !
Thank you so much !
... View more
09-28-2016
06:22 AM
@Ramesh Mani thank you for replying.
I don't understand why i need to use OS level group mapping for hive. The users do not exist on os and I'm not using the ranger for HDFS permissions. I'm using Ranger Just for granting hive permissions for users from Active Directory LDAP.
The users are not created at OS level. They do not exist on the os - they exist only in Active Directory.
Also, Ranger is able to see each users' groups from active directory (please see screenshot): I just want to use those Active Directory groups for granting permissions - instead of users. To sum up:
Ranger is synced with LDAP.
Users DO NOT exist on OS.
Ranger sees users from LDAP and their corresponding groups. (as in screenshot).
I am able to grant LDAP users permissions for hive tables.
LDAP users can access hive tables according to the permissions i give them.
BUT - if i'm using LDAP groups (which ranger sees) for granting permissions - LDAP users that reside in that group receive permission denied.
... View more
09-27-2016
04:05 PM
@Terry Stebbens Thank you for replying !
It is as if the groups are not even being checked - just the user.
When i try to select a table which my group has access to the hiveserver2 log shows: 2016-09-27 18:57:01,331 ERROR [HiveServer2-Handler-Pool: Thread-45]: ql.Driver (SessionState.java:printError(960)) - FAILED: HiveAccessControlException Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber]
org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber]
at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.checkPrivileges(RangerHiveAuthorizer.java:352)
at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:779)
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:574)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745) 2016-09-27 18:57:01,365 WARN [HiveServer2-Handler-Pool: Thread-45]: thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(492)) - Error executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber]
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:112)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber]
at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.checkPrivileges(RangerHiveAuthorizer.java:352)
at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:779)
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:574)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
... 15 more
... View more
09-27-2016
03:26 PM
1 Kudo
Hello We use HDP 2.3 with Ranger 0.5 for managing HIVE permissions. (not for HDFS. Just hive).
Ranger (usersync) is configured to use Active Directory and it syncs the users & groups from AD without any issues.
In Ranger >>> Settings >>> Users/Groups we see each user and it's corresponding Active Directory groups.
When granting authorizations to users for hive access - it works perfect !
However - when using groups (which is way more efficient to manage) it just doesn't work. Permission is always denied.
It seems using groups just doesn't work - only users.
Again - the sync works and i'm able to see the each users' groups in Ranger - but when i use groups instead of users for hive permissions - it does nothing.
Any ideas why ? Thanks in advance ! Adi J.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Ranger
09-13-2016
02:48 PM
Thank you !
It was the missing link !
... View more
09-13-2016
02:07 PM
1 Kudo
Hello We've upgraded Ambari from 2.1.2.1 to 2.4.0.1 according to the the upgrade guide
http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.0.1/bk_ambari-upgrade/content/ambari_upgrade_guide.html The entire process was successful and was performed according to the guide. After starting the Ambari i'm able to login and there is a restart indicator near every service (according to point 14 in the guide)
http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.0.1/bk_ambari-upgrade/content/upgrade_ambari.html However - the services won't start \ restart at all.
Every service fails with the same error:
resource_management.core.exceptions.Fail:
Execution of 'ambari-python-wrap /usr/bin/conf-select set-conf-dir --package
hadoop --stack-version 2.3.2.0-2950 --conf-version 0' returned 1.
/usr/hdp/2.3.2.0-2950/hadoop/conf does not exist Error:
Error: Unable to run the custom hook script ['/usr/bin/python',
'/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py',
'ANY', '/var/lib/ambari-agent/data/command-2602.json',
'/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY',
'/var/lib/ambari-agent/data/structured-out-2602.json', 'INFO',
'/var/lib/ambari-agent/tmp'] Any ideas ? Is this a known bug i missed ??
... View more
Labels:
- Labels:
-
Apache Ambari
08-28-2016
06:23 AM
Thank you both so much for the response!
The querying part using the processors you recommended is pretty straightforward - the problem is - what to use after the querying ? How do i transform the data into csv or other format which i can insert into hadoop ?
There is no ConvertAvroToCSV processor and ConvertAvroToOrc is not available in the 1.0 Beta release. Where can i get the 1.0 GA release ??
... View more
08-25-2016
05:54 PM
2 Kudos
Hi all I'm using nifi 1.0 and need to export data from an Oracle rdbms and insert the data into an Orc table on hive. What is the best way to do that (using what processors) using nothing but NiFi? To be even more exact if I have the source data as csv then it's not a problem, however It seems that I have to use a custom script for exporting the data to csv from the rdbms. I'd rather use NiFi processors but can't find the suitable to do so. Any suggestions? Adi
... View more
Labels:
- Labels:
-
Apache NiFi
08-24-2016
05:54 AM
So if a processor fails with a yellow warning - i am not supposed to get an email ? it is considered as "retry" and not "failure"?
... View more
08-24-2016
05:50 AM
@Shishir Saxena
Thank you so much for the answer and examples - i will try it.
... View more