Member since
05-29-2018
5
Posts
2
Kudos Received
0
Solutions
07-11-2018
02:59 PM
For Kafka, swap space is probably safe to clear (though I wouldn't), but you should avoid Kafka using swap space. If you look at disk IO on a Kafka broker node, it should be almost all writes, read should come from page cache. Kafka was designed to be the only tenant on a node and runs best that way. This is why you will find recommendations that say Kafka should not share nodes with Zookeeper or other Hadoop components. It is not always possible to dedicate machines to Kafka, so take a look at the disk IO when Kafka is running under normal load, if it is all writes, you can probably shrink the page cache a bit so you do less/no swapping. If there are lots of reads, you may need more memory or more nodes (unless you are deliberately and routinely reading topics from the beginning, in which case disk reads are unavoidable). Can't help you with the zookeeper, I've never had reason to dig into zookeeper's internals, it has always just worked.
... View more
05-31-2018
02:19 PM
I think this solution would work from a technical perspective. However, I am not sure my users will accept the naming pattern limitation. I have been unable to find a example of anyone using the {OWNER} variable. @svenkat said how useful it was here https://community.hortonworks.com/questions/118123/hdp-26-allows-user-variable-in-ranger-policies-are.html but didn't provide an example of actually using it. Does it work?
... View more
05-30-2018
08:40 PM
1 Kudo
I have a database where all members of a group are allowed to create tables. When a user creates a table, only the creator should be allowed access to that table. I will create a new policy to allow other members of the group access to the new table after an approval business process. HDP 2.6.5, Hive and Ranger (I cannot use Atlas and tag based policies at this time) This policy allows members of the group to create tables, but the owner has no access after creating. resources(database:testdb, table:*, column:*)
policyItems (accesses:create,groups:testgroup) beeline as user1, a member of testgroup 0: jdbc:hive2://localhost:10000> use testdb;
No rows affected (2.509 seconds)
0: jdbc:hive2://localhost:10000> CREATE TABLE testtable (id STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION '/hive/data/testtable';
No rows affected (0.669 seconds)
0: jdbc:hive2://localhost:10000> show tables;
+------------+--+
| tab_name |
+------------+--+
| testtable |
+------------+--+
1 row selected (1.029 seconds)
0: jdbc:hive2://localhost:10000> describe testtable;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [user1] does not have [SELECT] privilege on[testdb/testtable/*] (state=42000,code=40000) I tried adding a policy item using the {OWNER} variable but had the same result. resources(database:testdb, table:*, column:*)
policyItems [(accesses:[select,update,drop,alter,index,lock,read,write] users:{OWNER}), (accesses:create, groups:testgroup)]
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Ranger