Ranger admin not starting

[#1]

Installed single node cluster on Centos 6 and enabled kerberos on cluster. ranger not starting..

stderr: /var/lib/ambari-agent/data/errors-311.txt

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 231, in <module>
    RangerAdmin().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 93, in start
    self.configure(env, upgrade_type=upgrade_type, setup_db=params.stack_supports_ranger_setup_db_on_start)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 120, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 135, in configure
    setup_ranger_db()
  File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 265, in setup_ranger_db
    user=params.unix_user,
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-python-wrap /usr/hdp/current/ranger-admin/dba_script.py -q' returned 1. 2019-01-09 10:12:14,831  [I] Running DBA setup script. QuiteMode:True
2019-01-09 10:12:14,832  [I] Using Java:/usr/jdk64/jdk1.8.0_112/bin/java
2019-01-09 10:12:14,832  [I] DB FLAVOR:MYSQL
2019-01-09 10:12:14,832  [I] DB Host:nn.hadoop.com
2019-01-09 10:12:14,832  [I] ---------- Verifying DB root password ---------- 
2019-01-09 10:12:14,833  [I] DBA root user password validated
2019-01-09 10:12:14,833  [I] ---------- Verifying Ranger Admin db user password ---------- 
2019-01-09 10:12:14,833  [I] admin user password validated
2019-01-09 10:12:14,834  [I] ---------- Creating Ranger Admin db user ---------- 
2019-01-09 10:12:14,834  [JISQL] /usr/jdk64/jdk1.8.0_112/bin/java  -cp /usr/hdp/2.6.5.1050-37/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://nn.hadoop.com/mysql -u ranger -p '********' -noheader -trim -c \; -query "SELECT version();"
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user 'ranger'@'%' to database 'mysql' ErrorCode: 1044
2019-01-09 10:12:16,281  [E] Can't establish db connection.. Exiting..

stdout: /var/lib/ambari-agent/data/output-311.txt

2019-01-09 10:12:13,000 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=2.6.5.1050-37 -> 2.6.5.1050-37
2019-01-09 10:12:13,016 - Using hadoop conf dir: /usr/hdp/2.6.5.1050-37/hadoop/conf
2019-01-09 10:12:13,218 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=2.6.5.1050-37 -> 2.6.5.1050-37
2019-01-09 10:12:13,218 - Using hadoop conf dir: /usr/hdp/2.6.5.1050-37/hadoop/conf
2019-01-09 10:12:13,219 - Group['ranger'] {}
2019-01-09 10:12:13,233 - Group['hdfs'] {}
2019-01-09 10:12:13,234 - Group['hadoop'] {}
2019-01-09 10:12:13,234 - Group['users'] {}
2019-01-09 10:12:13,234 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-01-09 10:12:13,235 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-01-09 10:12:13,236 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None}
2019-01-09 10:12:13,236 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['ranger'], 'uid': None}
2019-01-09 10:12:13,237 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs'], 'uid': None}
2019-01-09 10:12:13,237 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-01-09 10:12:13,238 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-01-09 10:12:13,239 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2019-01-09 10:12:13,266 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2019-01-09 10:12:13,287 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2019-01-09 10:12:13,288 - Group['hdfs'] {}
2019-01-09 10:12:13,289 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hdfs']}
2019-01-09 10:12:13,289 - FS Type: 
2019-01-09 10:12:13,290 - Directory['/etc/hadoop'] {'mode': 0755}
2019-01-09 10:12:13,303 - File['/usr/hdp/2.6.5.1050-37/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'root', 'group': 'hadoop'}
2019-01-09 10:12:13,329 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2019-01-09 10:12:13,348 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2019-01-09 10:12:13,469 - Skipping Execute[('setenforce', '0')] due to not_if
2019-01-09 10:12:13,469 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'}
2019-01-09 10:12:13,535 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'}
2019-01-09 10:12:13,535 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'}
2019-01-09 10:12:13,563 - File['/usr/hdp/2.6.5.1050-37/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'root'}
2019-01-09 10:12:13,576 - File['/usr/hdp/2.6.5.1050-37/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'root'}
2019-01-09 10:12:13,583 - File['/usr/hdp/2.6.5.1050-37/hadoop/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2019-01-09 10:12:13,606 - File['/usr/hdp/2.6.5.1050-37/hadoop/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2019-01-09 10:12:13,607 - File['/usr/hdp/2.6.5.1050-37/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2019-01-09 10:12:13,622 - File['/usr/hdp/2.6.5.1050-37/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2019-01-09 10:12:13,655 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop', 'mode': 0644}
2019-01-09 10:12:13,678 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2019-01-09 10:12:13,682 - Testing the JVM's JCE policy to see it if supports an unlimited key length.
2019-01-09 10:12:14,007 - The unlimited key JCE policy is required, and appears to have been installed.
2019-01-09 10:12:14,397 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=2.6.5.1050-37 -> 2.6.5.1050-37
2019-01-09 10:12:14,402 - File['/usr/hdp/current/ranger-admin/ews/lib/mysql-connector-java.jar'] {'action': ['delete']}
2019-01-09 10:12:14,403 - Deleting File['/usr/hdp/current/ranger-admin/ews/lib/mysql-connector-java.jar']
2019-01-09 10:12:14,466 - File['/var/lib/ambari-agent/tmp/mysql-connector-java.jar'] {'content': DownloadSource('http://nn.hadoop.com:8080/resources/mysql-connector-java.jar'), 'mode': 0644}
2019-01-09 10:12:14,466 - Not downloading the file from http://nn.hadoop.com:8080/resources/mysql-connector-java.jar, because /var/lib/ambari-agent/tmp/mysql-connector-java.jar already exists
2019-01-09 10:12:14,498 - Execute[('cp', '--remove-destination', '/var/lib/ambari-agent/tmp/mysql-connector-java.jar', '/usr/hdp/2.6.5.1050-37/ranger-admin/ews/lib')] {'path': ['/bin', '/usr/bin/'], 'sudo': True}
2019-01-09 10:12:14,520 - File['/usr/hdp/2.6.5.1050-37/ranger-admin/ews/lib/mysql-connector-java.jar'] {'mode': 0644}
2019-01-09 10:12:14,521 - ModifyPropertiesFile['/usr/hdp/2.6.5.1050-37/ranger-admin/install.properties'] {'owner': 'ranger', 'properties': ...}
2019-01-09 10:12:14,565 - Modifying existing properties file: /usr/hdp/2.6.5.1050-37/ranger-admin/install.properties
2019-01-09 10:12:14,576 - File['/usr/hdp/2.6.5.1050-37/ranger-admin/install.properties'] {'owner': 'ranger', 'content': ..., 'group': None, 'mode': None, 'encoding': 'utf-8'}
2019-01-09 10:12:14,577 - Writing File['/usr/hdp/2.6.5.1050-37/ranger-admin/install.properties'] because contents don't match
2019-01-09 10:12:14,577 - ModifyPropertiesFile['/usr/hdp/2.6.5.1050-37/ranger-admin/install.properties'] {'owner': 'ranger', 'properties': {'SQL_CONNECTOR_JAR': '/usr/hdp/2.6.5.1050-37/ranger-admin/ews/lib/mysql-connector-java.jar'}}
2019-01-09 10:12:14,578 - Modifying existing properties file: /usr/hdp/2.6.5.1050-37/ranger-admin/install.properties
2019-01-09 10:12:14,579 - File['/usr/hdp/2.6.5.1050-37/ranger-admin/install.properties'] {'owner': 'ranger', 'content': ..., 'group': None, 'mode': None, 'encoding': 'utf-8'}
2019-01-09 10:12:14,579 - Writing File['/usr/hdp/2.6.5.1050-37/ranger-admin/install.properties'] because contents don't match
2019-01-09 10:12:14,581 - ModifyPropertiesFile['/usr/hdp/current/ranger-admin/install.properties'] {'owner': 'ranger', 'properties': {'audit_store': 'solr'}}
2019-01-09 10:12:14,581 - Modifying existing properties file: /usr/hdp/current/ranger-admin/install.properties
2019-01-09 10:12:14,582 - File['/usr/hdp/current/ranger-admin/install.properties'] {'owner': 'ranger', 'content': ..., 'group': None, 'mode': None, 'encoding': 'utf-8'}
2019-01-09 10:12:14,582 - Setting up Ranger DB and DB User
2019-01-09 10:12:14,582 - Execute['ambari-python-wrap /usr/hdp/current/ranger-admin/dba_script.py -q'] {'logoutput': True, 'environment': {'RANGER_ADMIN_HOME': '/usr/hdp/current/ranger-admin', 'JAVA_HOME': '/usr/jdk64/jdk1.8.0_112'}, 'user': 'ranger'}
2019-01-09 10:12:14,831  [I] Running DBA setup script. QuiteMode:True
2019-01-09 10:12:14,832  [I] Using Java:/usr/jdk64/jdk1.8.0_112/bin/java
2019-01-09 10:12:14,832  [I] DB FLAVOR:MYSQL
2019-01-09 10:12:14,832  [I] DB Host:nn.hadoop.com
2019-01-09 10:12:14,832  [I] ---------- Verifying DB root password ---------- 
2019-01-09 10:12:14,833  [I] DBA root user password validated
2019-01-09 10:12:14,833  [I] ---------- Verifying Ranger Admin db user password ---------- 
2019-01-09 10:12:14,833  [I] admin user password validated
2019-01-09 10:12:14,834  [I] ---------- Creating Ranger Admin db user ---------- 
2019-01-09 10:12:14,834  [JISQL] /usr/jdk64/jdk1.8.0_112/bin/java  -cp /usr/hdp/2.6.5.1050-37/ranger-admin/ews/lib/mysql-connector-java.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://nn.hadoop.com/mysql -u ranger -p '********' -noheader -trim -c \; -query "SELECT version();"
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user 'ranger'@'%' to database 'mysql' ErrorCode: 1044
2019-01-09 10:12:16,281  [E] Can't establish db connection.. Exiting..

Command failed after 1 tries

[#2 · Accepted solution]

@Lokesh Mukku

Your error indicates that the user ranger '@'% does not have access to 'mysql' DB. Hence you will need to properly grant the user.

SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user 'ranger'@'%' to database 'mysql' ErrorCode: 1044

Have you already performed the MySQL DB side prerequisite before using Ranger with MySQL? Like giving proper grant to "ranger" user ?

CREATE USER 'ranger'@'localhost' IDENTIFIED BY 'ranger';
GRANT ALL PRIVILEGES ON *.* TO 'ranger'@'localhost';
CREATE USER 'ranger'@'%' IDENTIFIED BY 'ranger';
GRANT ALL PRIVILEGES ON *.* TO 'ranger'@'%';
GRANT ALL PRIVILEGES ON *.* TO 'ranger'@'localhost' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO 'ranger'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;

.

Please refer to the following doc for more information: https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/configuring_...

For validating what kind of grants the user has, you can run the following SQL query on your mysql db

mysql> SELECT user, host FROM user WHERE user like 'range%' ;

.

[#3]

Above steps worked fine, but getting below error

Connection failed to http://nn.hadoop.com:6080/login.jsp (Execution of 'curl --location-trusted -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/db72a0e6-ec32-46b8-b2c9-174e417b2c4e -c /var/lib/ambari-agent/tmp/cookies/db72a0e6-ec32-46b8-b2c9-174e417b2c4e -w '%{http_code}' http://nn.hadoop.com:6080/login.jsp --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmpMOa5kZ 2>/tmp/tmpxYQlBP' returned 7. curl: (7) couldn't connect to host
000)

[#4]

@Lokesh Mukku

Means Ranger DB test connection passed however Ranger Admin component could not come up hence the Curl Call to

http://nn.hadoop.com:6080/login.jsp URL is failing.

So please check the Ranger Admin logs to find any error.