Member since
04-03-2019
962
Posts
1743
Kudos Received
146
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
10519 | 03-08-2019 06:33 PM | |
4635 | 02-15-2019 08:47 PM | |
3983 | 09-26-2018 06:02 PM | |
10182 | 09-07-2018 10:33 PM | |
5253 | 04-25-2018 01:55 AM |
11-25-2015
11:47 AM
4 Kudos
We can find it from regionserver logs but I want to know if there is any other better way.
... View more
Labels:
- Labels:
-
Apache HBase
11-24-2015
04:09 AM
Great! I believe ambari and hdfs services restart is needed if we delete configs using configs.sh script.
... View more
11-23-2015
10:37 AM
12 Kudos
I was working on one of the case and came across an issue where balancer utility was not working in namenode HA environment. HDP Version - 2.3.2.0 Ambari Version - 2.1.2 Issue - balancer was failing with below error: [hdfs@slave1 ~]$ hdfs balancer 15/11/20 11:46:10 INFO balancer.Balancer: namenodes = [hdfs://mycluster, hdfs://mycluster:8020] 15/11/20 11:46:10 INFO balancer.Balancer: parameters = Balancer.Parameters [BalancingPolicy.Node, threshold = 10.0, max idle iteration = 5, #excluded nodes = 0, #included nodes = 0, #source nodes = 0, run during upgrade = false] [..] 15/11/20 11:46:11 INFO block.BlockTokenSecretManager: Setting block keys 15/11/20 11:46:11 INFO balancer.KeyManager: Update block keys every 2hrs, 30mins, 0sec java.io.IOException: Another Balancer is running.. Exiting ... Nov 20, 2015 11:46:11 AM Balancing took 1.883 seconds
Resolution(workaround) - 1. Copy /etc/hadoop/conf/* /home/hdfs/copied_conf/ 2. Edit /home/hdfs/copied_conf/hdfs-site.xml and delete below property <property>
<name>dfs.namenode.rpc-address</name>
<value>xyz:8020</value>
</property>
3. Run balancer with below command(as user hdfs):
hdfs --config /home/hdfs/copied_conf balancer
Permanent Solution - This will get fixed in Ambari 2.1.3. version (Refer - https://hortonworks.jira.com/browse/BUG-45749)
... View more
11-21-2015
03:17 PM
10 Kudos
Step 1 - Goto ambari UI and add/modify below properties Goto service hive → configs and change autherization to SQLStdAuth Step 2 - In Hive-site.xml, make sure you have set below properties: hive.server2.enable.doAs --> false
hive.users.in.admin.role --> root (comma separated list of users) Step 3 - Make sure that you have below properties set in Hiveserver2-site.xml hive.security.authorization.manager --> org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdConfOnlyAuthorizerFactory
hive.security.authorization.enabled --> true
hive.security.authenticator.manager --> org.apache.hadoop.hive.ql.security.ProxyUserAuthenticator Step 4 - Restart hive services from UI Step 5 - Testing our setup, in this we will create one readonly user and try to drop table.
5.1 Login to beeline using root ( as we have added root in hive.users.in.admin.role )
0: jdbc:hive2://localhost:10010> !connect jdbc:hive2://localhost:10010Connecting to jdbc:hive2://localhost:10010
Enter username for jdbc:hive2://localhost:10010: root
Enter password for jdbc:hive2://localhost:10010: ****
Connected to: Apache Hive (version 1.2.1.2.3.2.0-2950)Driver: Hive JDBC (version 1.2.1.2.3.2.0-2950)Transaction isolation:
TRANSACTION_REPEATABLE_READ1: jdbc:hive2://localhost:10010> 5.2 Now by default there is only one role - public, you need to run below command to set your role as ADMIN. 0: jdbc:hive2://localhost:10010> SHOW CURRENT ROLES;
+---------+--+
| role |
+---------+--+
| public |
+---------+--+ 5.3 Set role as admin for user root 1: jdbc:hive2://localhost:10010> set role ADMIN;
No rows affected (0.445 seconds)1:
jdbc:hive2://localhost:10010> show roles;
+---------+--+
| role |
+---------+--+
| admin |
| public |
+---------+--+
2 rows selected (0.165 seconds) 5.4 Create new role readonly and add readonly_user in that group 0: jdbc:hive2://slave1.hortonworks.com:10010/> create role readonly;
No rows affected (0.071 seconds) 5.5 Verify that new role has been created successfully 0: jdbc:hive2://slave1.hortonworks.com:10010/> show roles;
+-----------+--+
| role |
+-----------+--+
| admin |
| public |
| readonly |
+-----------+--+
3 rows selected (0.051 seconds)
0: jdbc:hive2://slave1.hortonworks.com:10010/> 5.6 Add readonly_user into readonly role
5: jdbc:hive2://slave1.hortonworks.com:10010> grant role readonly to user readonly_user;
No rows affected (0.088 seconds)
5: jdbc:hive2://slave1.hortonworks.com:10010>
5.7 Grant select privileges to role readonly 5: jdbc:hive2://slave1.hortonworks.com:10010> grant select on table batting to role readonly;
No rows affected (0.405 seconds)
5: jdbc:hive2://slave1.hortonworks.com:10010> 5.8 Verify grants for role readonly 0: jdbc:hive2://slave1.hortonworks.com:10010/> show grant role readonly;
+-----------+----------+------------+---------+-----------------+-----------------+------------+---------------+----------------+----------+--+
| database | table | partition | column | principal_name | principal_type | privilege | grant_option | grant_time | grantor |
+-----------+----------+------------+---------+-----------------+-----------------+------------+---------------+----------------+----------+--+
| default | batting | | | readonly | ROLE | SELECT | false | 1447877696000 | root |
+-----------+----------+------------+---------+-----------------+-----------------+------------+---------------+----------------+----------+--+
1 row selected (0.06 seconds) 5.9 Now login to beeline by user readonly_user and try to drop table batting beeline> !connect jdbc:hive2://slave1.hortonworks.com:10010/
Connecting to jdbc:hive2://slave1.hortonworks.com:10010/
Enter username for jdbc:hive2://slave1.hortonworks.com:10010/: readonly_user
Enter password for jdbc:hive2://slave1.hortonworks.com:10010/: ********
Connected to: Apache Hive (version 1.2.1.2.3.2.0-2950)Driver: Hive JDBC (version 1.2.1.2.3.2.0-2950)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://slave1.hortonworks.com:10010/> drop table batting;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: Principal [name=readonly_user, type=USER] does not have following privileges for operation DROPTABLE [[OBJECT OWNERSHIP] on Object [type=TABLE_OR_VIEW, name=default.batting]] (state=42000,code=40000)
Note - we are getting an error here because readonly_user does not have permission to drop table batting! 5.10 Let's try to access some rows from table batting 0: jdbc:hive2://slave1.hortonworks.com:10010/> select * from batting limit 5;
+--------------------+---------------+---------------+--+
| batting.player_id | batting.year | batting.runs |
+--------------------+---------------+---------------+--+
| playerID | NULL | NULL |
| aardsda01 | 2004 | 0 |
| aardsda01 | 2006 | 0 |
| aardsda01 | 2007 | 0 |
| aardsda01 | 2008 | 0 |
+--------------------+---------------+---------------+--+
5 rows selected (0.775 seconds)
0: jdbc:hive2://slave1.hortonworks.com:10010/>
We can see that grants are working and user can see the contents but cannot delete the table. Attached my hive-site.xml for more details.
... View more
Labels:
11-21-2015
01:55 PM
1 Kudo
Thank you so much @Neeraj Sabharwal
... View more
11-21-2015
01:55 PM
1 Kudo
Thank you so much @sneethiraj 🙂
... View more
11-20-2015
02:01 PM
mysql-5.1.73
... View more
- « Previous
- Next »