question Re: Cannot start Ranger admin process with mysql in Archives of Support Questions (Read Only)

Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:46 GMT

Ok, I seem to have hive up and running with mysql, but now I cannot get ranger admin to start:

stderr:   /var/lib/ambari-agent/data/errors-1023.txt

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 155, in <module>
    RangerAdmin().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 512, in restart
    self.stop(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 54, in stop
    Execute(format('{params.ranger_stop}'), environment={'JAVA_HOME': params.java_home}, user=params.unix_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/ranger-admin-stop' returned 126. -bash: /usr/bin/ranger-admin-stop: Permission denied

=============================

stdout:   /var/lib/ambari-agent/data/output-1023.txt

2016-10-25 10:58:39,550 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.4.2.0-258
2016-10-25 10:58:39,550 - Checking if need to create versioned conf dir /etc/hadoop/2.4.2.0-258/0
2016-10-25 10:58:39,550 - call['conf-select create-conf-dir --package hadoop --stack-version 2.4.2.0-258 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-10-25 10:58:39,587 - call returned (1, '/etc/hadoop/2.4.2.0-258/0 exist already', '')
2016-10-25 10:58:39,588 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.4.2.0-258 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-10-25 10:58:39,616 - checked_call returned (0, '')
2016-10-25 10:58:39,616 - Ensuring that hadoop has the correct symlink structure
2016-10-25 10:58:39,617 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-10-25 10:58:39,736 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.4.2.0-258
2016-10-25 10:58:39,736 - Checking if need to create versioned conf dir /etc/hadoop/2.4.2.0-258/0
2016-10-25 10:58:39,737 - call['conf-select create-conf-dir --package hadoop --stack-version 2.4.2.0-258 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-10-25 10:58:39,763 - call returned (1, '/etc/hadoop/2.4.2.0-258/0 exist already', '')
2016-10-25 10:58:39,764 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.4.2.0-258 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-10-25 10:58:39,791 - checked_call returned (0, '')
2016-10-25 10:58:39,792 - Ensuring that hadoop has the correct symlink structure
2016-10-25 10:58:39,792 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-10-25 10:58:39,794 - Group['kms'] {}
2016-10-25 10:58:39,795 - Group['spark'] {}
2016-10-25 10:58:39,796 - Group['ranger'] {}
2016-10-25 10:58:39,797 - Group['hadoop'] {}
2016-10-25 10:58:39,797 - Group['users'] {}
2016-10-25 10:58:39,800 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,801 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,802 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,803 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['ranger']}
2016-10-25 10:58:39,804 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2016-10-25 10:58:39,804 - User['kms'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,805 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,806 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2016-10-25 10:58:39,807 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,808 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,809 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,809 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,810 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-10-25 10:58:39,811 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-10-25 10:58:39,814 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2016-10-25 10:58:39,819 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2016-10-25 10:58:39,820 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2016-10-25 10:58:39,821 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-10-25 10:58:39,822 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2016-10-25 10:58:39,827 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if
2016-10-25 10:58:39,828 - Group['hdfs'] {}
2016-10-25 10:58:39,828 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'hdfs']}
2016-10-25 10:58:39,829 - FS Type: 
2016-10-25 10:58:39,829 - Directory['/etc/hadoop'] {'mode': 0755}
2016-10-25 10:58:39,847 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2016-10-25 10:58:39,848 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777}
2016-10-25 10:58:39,863 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2016-10-25 10:58:39,881 - Directory['/var/log/hadoop'] {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2016-10-25 10:58:39,883 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'}
2016-10-25 10:58:39,883 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'}
2016-10-25 10:58:39,889 - File['/usr/hdp/current/hadoop-client/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2016-10-25 10:58:39,891 - File['/usr/hdp/current/hadoop-client/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2016-10-25 10:58:39,892 - File['/usr/hdp/current/hadoop-client/conf/log4j.properties'] {'content': ..., 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2016-10-25 10:58:39,909 - File['/usr/hdp/current/hadoop-client/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs', 'group': 'hadoop'}
2016-10-25 10:58:39,910 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2016-10-25 10:58:39,911 - File['/usr/hdp/current/hadoop-client/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2016-10-25 10:58:39,917 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2016-10-25 10:58:39,922 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2016-10-25 10:58:40,092 - Execute['/usr/bin/ranger-admin-stop'] {'environment': {'JAVA_HOME': '/usr/jdk64/jdk1.8.0_60'}, 'user': 'ranger'}

I created the database and user accounts in mysql it needs, but I don't see why it would give a "permission denied" error attempting to stop ranger-admin. I tried running the script manually from the root account, to no avail.

Re: Cannot start Ranger admin process with mysql

bhagan — Tue, 01 Nov 2016 00:26:47 GMT

Could you provide us the output from the ls -la command for /usr/bin/ranger-admin-stop?

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

@Gary Cameron

Whats the output of below command

# hostname -f

From MySQL prompt

Mysql>show grants;

Then run this grants statement after connect like root

mysql> use rangerdb;
mysql> GRANT ALL PRIVILEGES ON * . * TO 'rangerdb'@'%'; 
mysql> Flush privileges;
mysql> quit;

And retry it looks a privilege issue

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

The below statement is wrong though it runn successfully. Notice the .* after rangerdb

mysql> GRANT ALL PRIVILEGES ON rangerdb.* TO 'ranger'@'nn.example.com' IDENTIFIED BY 'rangerdba';

The below statement should be run

mysql> GRANT ALL PRIVILEGES ON rangerdb TO 'ranger'@'nn.example.com' IDENTIFIED BY 'rangerdba';

Then retry

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

@Gary Cameron

You can restart your Ranger setup with the attached a customized document. You will need to ensure you are ROOT when creating the user ranger or whatever user with corresponding password of your choice. Then log on as ranger to create the ranger database then exit and continue through the Ambari UI you should be fine.

Happy Hadooping

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

@Gary Cameron

Any updates on this ranger problem.

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

@Gary Cameron

It doesn't hurt ,this is a sample script but your setup a ranger database using a script. You could create another Db and try the ranger setup. It could be a privilege issue ..Guess work! I am providing you 2 methods to help out

CREATE USER 'rangeradmin'@'%' IDENTIFIED BY 'xxxx';
GRANT ALL PRIVILEGES ON *.* TO 'rangeradmin'@'%';
CREATE USER 'rangeradmin'@'localhost' IDENTIFIED BY 'xxxx';
GRANT ALL PRIVILEGES ON *.* TO 'rangeradmin'@'localhost';
CREATE USER 'rangeradmin'@'FQDN' IDENTIFIED BY 'xxxx';
GRANT ALL PRIVILEGES ON *.* TO 'rangeradmin'@'FQDN';
FLUSH PRIVILEGES;
Log on to MySQL as ranger  
create database rangerdb;

As root user navigate to the below directory

# /usr/hdp/2.4.2.0-258/ranger-admin 
#./setup.sh
# ./set_globals.sh

Restart the ranger admin from the ambari GUI and it should fire up.

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:47 GMT

I manually fixed the symlink by looking at another working cluster. No idea how or why it got confused like that.

[root@bu-hdp2-nn ranger-usersync]# ln -s /etc/ranger-usersync/2.4.2.0-258/0 conf[root@bu-hdp2-nn ranger-usersync]# ls -l conflrwxrwxrwx. 1 root root 34 Oct 28 10:35 conf -> /etc/ranger-usersync/2.4.2.0-258/0[root@bu-hdp2-nn ranger-usersync]# pwd/usr/hdp/current/ranger-usersync

And Ranger services seem to running now.

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

@Gary Cameron

Can you copy and paste in here your ranger database user create script?

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

@Gary Cameron

I am happy it resolved your issues.....and welcome to HCC

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:47 GMT

@bhagan: Thanks for your response!

[root@bu-hdp2-nn ~]# ls -la /usr/bin/ranger-admin-stoplrwxrwxrwx. 1 root root 51 Jun 14 15:42 /usr/bin/ranger-admin-stop -> /usr/hdp/current/ranger-admin/ews/ranger-admin-stop[root@bu-hdp2-nn ~]

# ls -la  /usr/hdp/current/ranger-admin/ews/ranger-admin-stoplrwxrwxrwx. 1 root root 58 Oct 25 09:54 /usr/hdp/current/ranger-admin/ews/ranger-admin-stop -> /usr/hdp/2.4.2.0-258/ranger-admin/ews/stop-ranger-admin.sh

Everything looks ok to me from here...

Re: Cannot start Ranger admin process with mysql

Shelton — Tue, 01 Nov 2016 00:26:47 GMT

@Gary Cameron

Whats the output of below command

# hostname -f

From MySQL prompt

Mysql>show grants;

Then run this grants statement after connect like root

mysql> use rangerdb;
mysql> GRANT ALL PRIVILEGES ON * . * TO 'ranger'@'%'; 
mysql> Flush privileges;
mysql> quit;

And retry it looks a privilege issue

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:48 GMT

Ok, I did everything:

[root@bu-hdp2-nn ranger-admin]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 170
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>
mysql> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  -> ;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.' at line 1
mysql> GRANT ALL PRIVILEGES ON rangerdb TO 'ranger'@'nn.example.com' IDENTIFIED BY 'rangerdba';
ERROR 1046 (3D000): No database selected
mysql> use rangerdb;
Database changed
mysql> GRANT ALL PRIVILEGES ON rangerdb TO 'ranger'@'nn.example.com' IDENTIFIED BY 'rangerdba';
Query OK, 0 rows affected (0.02 sec)

mysql> show grants
  -> ;
+----------------------------------------------------------------------------------------------------------------------------------------+
| Grants for root@localhost  |
+----------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY PASSWORD 'xxxxxxxxx' WITH GRANT OPTION |
+----------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> quit
Bye

I still have that last problem:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py", line 78, in <module>
    RangerUsersync().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 530, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py", line 50, in start
    self.configure(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py", line 44, in configure
    ranger('ranger_usersync', upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 44, in ranger
    setup_usersync(upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 336, in setup_usersync
    owner = params.unix_user
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 156, in action_create
    raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname))
resource_management.core.exceptions.Fail: Applying Directory['/usr/hdp/current/ranger-usersync/conf/'] failed, parent directory /usr/hdp/current/ranger-usersync/conf doesn't exist

[root@bu-hdp2-nn ranger-usersync]# ls -l /usr/hdp/current/ranger-usersync/conf
lrwxrwxrwx. 1 root root 25 Oct 25 09:55 /usr/hdp/current/ranger-usersync/conf -> /etc/ranger/usersync/conf
[root@bu-hdp2-nn ranger-usersync]# ls -l /etc/ranger/usersync/conf
lrwxrwxrwx. 1 ranger ranger 37 Sep  1 13:54 /etc/ranger/usersync/conf -> /usr/hdp/current/ranger-usersync/conf
[root@bu-hdp2-nn ranger-usersync]# ls -l /usr/hdp/current/ranger-usersync/conf
lrwxrwxrwx. 1 root root 25 Oct 25 09:55 /usr/hdp/current/ranger-usersync/conf -> /etc/ranger/usersync/conf

Wow, more circular symlinks! I'm not sure how my ranger install got itself so borked.

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:48 GMT

Thanks for continuing to look at this. I think I am almost there, although it's been painful.

[root@bu-hdp2-nn ranger-admin]# hostname -f
nn.example.com

[root@bu-hdp2-nn ranger-admin]# mysql -u ranger -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 166
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show grants;
+------------------------------------------------------------------------------------------------------------------------------------------+
| Grants for ranger@localhost  |
+------------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'ranger'@'localhost' IDENTIFIED BY PASSWORD 'xxxxxxxxx' WITH GRANT OPTION |
+------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> quit
Bye

[root@bu-hdp2-nn ranger-admin]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 168
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use rangerdb;
Database changed
mysql> GRANT ALL PRIVILEGES ON * . * TO 'ranger'@'%';
Query OK, 0 rows affected (0.00 sec)

mysql> Flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> quit
Bye
[root@bu-hdp2-nn ranger-admin]#

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:48 GMT

Sorry about the spam, made it a bit further on my own:

mysql> GRANT ALL PRIVILEGES ON rangerdb.* TO ranger@nn.example.com IDENTIFIED BY 'rangerdba';
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-nn.example.com IDENTIFIED BY 'rangerdba'' at line 1 
mysql> GRANT ALL PRIVILEGES ON rangerdb.* TO 'ranger'@'nn.example.com' IDENTIFIED BY 'rangerdba';

Query OK, 0 rows affected (0.00 sec)

It now only fails to start the last component:

stderr:   /var/lib/ambari-agent/data/errors-1160.txt
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py", line 78, in <module>
    RangerUsersync().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 530, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py", line 50, in start
    self.configure(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py", line 44, in configure
    ranger('ranger_usersync', upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 44, in ranger
    setup_usersync(upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 336, in setup_usersync
    owner = params.unix_user
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 156, in action_create
    raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname))
resource_management.core.exceptions.Fail: Applying Directory['/usr/hdp/current/ranger-usersync/conf/'] failed, parent directory /usr/hdp/current/ranger-usersync/conf doesn't exist

Need to understand why the directory it is looking for doesn't exist, and what I need to do to recreate it.

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:48 GMT

Ok, I compared with another system and manually fixed the symlink:

[root@bu-hdp2-nn ranger-admin]# ln -s /etc/ranger-admin/2.4.2.0-258/0 conf 
[root@bu-hdp2-nn ranger-admin]# ls -l conf
lrwxrwxrwx. 1 root root 31 Oct 27 15:36 conf -> /etc/ranger-admin/2.4.2.0-258/0

Now when I restart the error is different:

stderr:   /var/lib/ambari-agent/data/errors-1154.txt
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 155, in <module>
    RangerAdmin().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 530, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 75, in start
    self.configure(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 102, in configure
    ranger('ranger_admin', upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 41, in ranger
    setup_ranger_admin(upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 80, in setup_ranger_admin
    Execute(db_connection_check_command, path='/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin', tries=5, try_sleep=10, environment=env_dict)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/jdk64/jdk1.8.0_60/bin/java -cp /usr/lib/ambari-agent/DBConnectionVerification.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/current/ranger-admin/ews/lib/* org.apache.ambari.server.DBConnectionVerification 'jdbc:mysql://nn.example.com/rangerdb' ranger [PROTECTED] com.mysql.jdbc.Driver' returned 1. ERROR: Unable to connect to the DB. Please check DB connection properties.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user 'ranger'@'nn.example.com' to database 'rangerdb'

I was sure I had set the password correctly, it works here:

[root@bu-hdp2-nn ranger-admin]# mysql -u ranger -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 151 
Server version: 5.1.73 Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. 
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. 
mysql> show databases
    -> ; 
+--------------------+ 
| Database           | 
+--------------------+ 
| information_schema | 
| hive               | 
| mysql              | 
| ranger             | 
| rangerdb           | 
+--------------------+ 
5 rows in set (0.00 sec)

Not sure why I would see access denied?

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:48 GMT

You've been very helpful!

I assume in the first section

Log on to MySQL as ranger

You actually meant:

Log on to MySQL as rangeradmin

Since this was the account created? Assuming this is what you meant, I followed the steps you suggested and created the database from here.

When I restarted ranger the error was a bit different, so maybe this is progress. But the circular symlinks complaint is weird.

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 155, in <module>
    RangerAdmin().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 530, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 75, in start
    self.configure(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 102, in configure
    ranger('ranger_admin', upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 41, in ranger
    setup_ranger_admin(upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 55, in setup_ranger_admin
    recursive = True
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 144, in action_create
    raise Fail("Applying %s failed, looped symbolic links found while resolving %s" % (self.resource, path))
resource_management.core.exceptions.Fail: Applying Directory['/usr/hdp/current/ranger-admin/conf'] failed, looped symbolic links found while resolving /usr/hdp/current/ranger-admin/conf

I tried your second suggestion and this was the result:

[root@bu-hdp2-nn ranger-admin]# ./setup.sh
2016-10-27 09:01:51,841  --------- Running Ranger PolicyManager Web Application Install Script ---------
2016-10-27 09:01:51,843  [I] uname=Linux
2016-10-27 09:01:51,846  [I] hostname=nn.example.com
2016-10-27 09:01:51,852  [I] DB_FLAVOR=MYSQL
2016-10-27 09:01:51,853  [I] Checking distribution name..
2016-10-27 09:01:51,864  [I] Found distribution : CentOS
2016-10-27 09:01:51,866  [I] check if command /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java exists
2016-10-27 09:01:51,868  [I] '/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java' command found
2016-10-27 09:01:51,974  [I] Checking MYSQL CONNECTOR FILE : /usr/share/java/mysql-connector-java.jar
2016-10-27 09:01:51,976  [I] MYSQL CONNECTOR FILE : /usr/share/java/mysql-connector-java.jar file found
2016-10-27 09:01:51,978  [I] Setting up UNIX user : ranger and group: ranger
groupadd: group 'ranger' already exists
2016-10-27 09:01:52,001  [I] User already exists, adding it to group
usermod: no changes
2016-10-27 09:01:52,007  [I] Setting up UNIX user : ranger and group: ranger DONE
2016-10-27 09:01:52,008  [I] Setting up installation files and directory
2016-10-27 09:01:52,012  [I] Copying /usr/hdp/2.4.2.0-258/ranger-admin/ews/webapp/WEB-INF/classes/conf.dist /usr/hdp/2.4.2.0-258/ranger-admin/ews/webapp/WEB-INF/classes/conf
mkdir: cannot create directory `/usr/hdp/2.4.2.0-258/ranger-admin/ews/webapp/WEB-INF/classes/conf': File exists
cp: accessing `/usr/hdp/2.4.2.0-258/ranger-admin/ews/webapp/WEB-INF/classes/conf': Too many levels of symbolic links
2016-10-27 09:01:52,018  [I] Setting up init.d
2016-10-27 09:01:52,022  [I] Creating script S88ranger-admin/K90ranger-admin in /etc/rc2.d directory ....
2016-10-27 09:01:52,027  [I] Creating script S88ranger-admin/K90ranger-admin in /etc/rc3.d directory ....
2016-10-27 09:01:52,031  [I] Creating script S88ranger-admin/K90ranger-admin in /etc/rc.d/rc2.d directory ....
2016-10-27 09:01:52,036  [I] Creating script S88ranger-admin/K90ranger-admin in /etc/rc.d/rc3.d directory ....
2016-10-27 09:01:52,087  [I] Setting up installation files and directory DONE
2016-10-27 09:01:52,115  [I] /usr/hdp/2.4.2.0-258/ranger-admin/ews/webapp folder found
2016-10-27 09:01:52,117  [I] db/mysql/xa_core_db.sql file found
2016-10-27 09:01:52,118  [I] Copying MYSQL Connector to /usr/hdp/2.4.2.0-258/ranger-admin/ews/webapp/WEB-INF/lib
2016-10-27 09:01:52,124  [I] Copying MYSQL Connector to /usr/hdp/2.4.2.0-258/ranger-admin/ews/webapp/WEB-INF/lib DONE
2016-10-27 09:01:52,128  [I] check if command python exists
2016-10-27 09:01:52,130  [I] 'python' command found
2016-10-27 09:01:52,136  [E] 'setup_mode' not found in /usr/hdp/2.4.2.0-258/ranger-admin/install.properties file while getting....!!
2016-10-27 09:01:52,138  [I] Setup mode is not set. Running DBA steps..
2016-10-27 09:01:52,201  [I] Running DBA setup script. QuiteMode:True
2016-10-27 09:01:52,201  [I] Using Java:/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
2016-10-27 09:01:52,201  [I] DB FLAVOR:MYSQL
2016-10-27 09:01:52,201  [I] DB Host:localhost
2016-10-27 09:01:52,202  [I] ---------- Verifing DB root password ----------
2016-10-27 09:01:52,202  [I] DBA root user password validated
2016-10-27 09:01:52,202  [I] ---------- Verifing Ranger Admin db user password ----------
2016-10-27 09:01:52,202  [E] Blank password is not allowed,please enter valid password.

I certainly didn't set it blank in the first set of steps you gave. Must be using a different default?

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:48 GMT

Sorry, I am not using a script, is there a default one? I just added the mysql accounts manually.

Here is my configuration: The test connection works fine, but when I start ranger it fails with the message above?

Re: Cannot start Ranger admin process with mysql

gary_cameron — Tue, 01 Nov 2016 00:26:48 GMT

See response below

Re: Cannot start Ranger admin process with mysql

anand_ranganath — Thu, 05 Jan 2017 07:00:18 GMT

I bumped into the same problem and your solution helped me to resolve the problem.

Thanks