Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Ranger Fails to start

Highlighted

Ranger Fails to start

New Contributor

Hi,

I have an issue where ranger will not start. The error log appears to show that ranger is attempting to install a number of java patches and is unable to do so when it hits one, J10008, which needs to be configured.

 

Note that we have checked and the appropriate user has permissions on the database to perform an update. 

 

If anyone has seen this before or has any ideas please let me know. Error log below.

 

 

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/RANGER/package/scripts/ranger_admin.py", line 241, in <module>
    RangerAdmin().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/RANGER/package/scripts/ranger_admin.py", line 97, in start
    self.configure(env, upgrade_type=upgrade_type, setup_db=params.stack_supports_ranger_setup_db_on_start)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/RANGER/package/scripts/ranger_admin.py", line 138, in configure
    setup_ranger_xml.setup_java_patch()
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/RANGER/package/scripts/setup_ranger_xml.py", line 304, in setup_java_patch
    user=params.unix_user,
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
    returns=self.resource.returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 316, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-python-wrap /usr/hdp/current/ranger-admin/db_setup.py -javapatch' returned 1. 2019-10-23 18:16:36,204  [I] DB FLAVOR :POSTGRES
2019-10-23 18:16:36,205  [I] --------- Verifying Ranger DB connection ---------
2019-10-23 18:16:36,205  [I] Checking connection
2019-10-23 18:16:36,206  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "SELECT 1;"
2019-10-23 18:16:36,634  [I] connection success
2019-10-23 18:16:36,635  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and inst_by = 'Ranger 1.2.0.3.1.0.0-78' and active = 'Y';"
2019-10-23 18:16:37,220  [I] ----------------- Applying java patches ------------
2019-10-23 18:16:37,224  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10001' and active = 'Y';"
2019-10-23 18:16:37,876  [I] Java patch PatchPasswordEncryption_J10001 is already applied
2019-10-23 18:16:37,877  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10002' and active = 'Y';"
2019-10-23 18:16:38,539  [I] Java patch PatchMigration_J10002 is already applied
2019-10-23 18:16:38,540  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10003' and active = 'Y';"
2019-10-23 18:16:39,230  [I] Java patch PatchPersmissionModel_J10003 is already applied
2019-10-23 18:16:39,230  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10004' and active = 'Y';"
2019-10-23 18:16:39,884  [I] Java patch PatchForServiceVersionInfo_J10004 is already applied
2019-10-23 18:16:39,885  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10005' and active = 'Y';"
2019-10-23 18:16:40,423  [I] Java patch PatchTagModulePermission_J10005 is already applied
2019-10-23 18:16:40,423  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10006' and active = 'Y';"
2019-10-23 18:16:41,037  [I] Java patch PatchForHiveServiceDefUpdate_J10006 is already applied
2019-10-23 18:16:41,037  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10007' and active = 'Y';"
2019-10-23 18:16:41,627  [I] Java patch PatchForHiveServiceDefUpdate_J10007 is already applied
2019-10-23 18:16:41,627  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10008' and active = 'Y';"
2019-10-23 18:16:42,219  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10008' and active = 'N';"
2019-10-23 18:16:42,786  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('J10008', current_timestamp, 'Ranger 1.2.0.3.1.0.0-78', current_timestamp, 'knox.bob.local','N') ;"
2019-10-23 18:16:43,382  [I] java patch PatchForTagServiceDefUpdate_J10008 is being applied..
[EL Severe]: 2019-10-23 18:17:25.484--ServerSession(315885065)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!
Error Code: 0
[EL Severe]: ejb: 2019-10-23 18:17:25.509--ServerSession(315885065)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!
Error Code: 0
[EL Severe]: 2019-10-23 18:17:57.248--ServerSession(315885065)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!
Error Code: 0
[EL Severe]: ejb: 2019-10-23 18:17:57.249--ServerSession(315885065)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!
Error Code: 0
2019-10-23 18:17:57,512  [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java  -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-jdbc.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://knox.bob.local/ranger -u rangeradmin -p '********' -noheader -trim -c \; -query "delete from x_db_version_h where version='J10008' and active='N' and updated_by='knox.bob.local';"
2019-10-23 18:17:58,130  [E] applying java patch PatchForTagServiceDefUpdate_J10008 failed

 

 

2 REPLIES 2

Re: Ranger Fails to start

Super Collaborator

Error shows that DB connection is lost while updating the table with patch information.

 

It seems script failed after multiple retries for the DB connection. Logs shows that  patch script was able to install other patches before the failure. Please troubleshoot from postgres DB, start by verifying the connections and postgres log. 

Re: Ranger Fails to start

New Contributor

Hi,

 

We have checked this the number of concurrent connections on the database appears to be plenty. I am not sure it does actually update any patches since all of the previous connections are selects not updates - hence my comment about we have checked the permissions. 

 

Thanks for your thoughts on this and please keep going!

Don't have an account?
Coming from Hortonworks? Activate your account here