Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Ranger Admin Unable to acquire connections to underlying database KmsKeyMgr.svcStore

Highlighted

Ranger Admin Unable to acquire connections to underlying database KmsKeyMgr.svcStore

Explorer

I've upgraded to ambari to 2.7.3.0. In addition to upgrading ambari, I upgraded postgres from v9 to v10. After upgrading postgres, I can no longer start the ranger admin service within ambari. I've been seeing the notification: "Ranger admin service is not reachable, please restart the service." When I try to restart the service I observe the following log in /var/log/ranger/admin/catalina.out (see stack trace below):

 

Any ideas on what is going on here? I have a feeling there is a conflict between v9/v10 postgres db possibly, but I'm not entirely sure on this.

 

Dec 02, 2019 2:11:13 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'kmsKeyMgr': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: org.apache.ranger.biz.ServiceDBStore org.apache.ranger.biz.KmsKeyMgr.svcStore; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'serviceDBStore': Invocation of init method failed; nested exception is org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20131113-a7346c6): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!
Error Code: 0
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:287)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:605)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:925)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:472)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5068)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5584)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1572)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1562)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: org.apache.ranger.biz.ServiceDBStore org.apache.ranger.biz.KmsKeyMgr.svcStore; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'serviceDBStore': Invocation of init method failed; nested exception is org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20131113-a7346c6): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!
Error Code: 0

2 REPLIES 2
Highlighted

Re: Ranger Admin Unable to acquire connections to underlying database KmsKeyMgr.svcStore

Mentor

@rwinters 

You could be right that your issue is incompatibility? Can you check exactly what version of Postgres you upgraded to?

It's usually advisable to check the Hortonworks support matrix  before launching any upgrade I only hope it's a dev environment  see screenshot

 

Postgres10.PNG

 Here you see PostgreSQL 10.7 is only compatible with Ambari 2.7.4 and HDP 3.1.4  very useful tool.

 

Happy hadooping 

Highlighted

Re: Ranger Admin Unable to acquire connections to underlying database KmsKeyMgr.svcStore

Explorer

Thanks for your response. Sorry for my the late response, but I wanted to pass this along in case someone runs into this issue. So I did some more testing and what I found is that I had an incompatible postgresql-jdbc driver. So I was upgrading from 9.6 to 10.x and it appears the jdbc driver I was using was pretty old and incompatible with version 10.x. Once I upgraded the postgresql-jdbc driver everything worked just fine. Here is what I did to resolve my issue:

 

  • wget https://jdbc.postgresql.org/download/postgresql-42.2.9.jar
  • mv /usr/share/java/postgresql-jdbc.jar /usr/share/java/postgresql-jdbc.jar.old  (backing up the old driver)
  • cp ~/postgresql-42.2.9.jar /usr/share/java/postgresql-jdbc.jar
  • ambari-server setup --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar

 

 

Don't have an account?
Coming from Hortonworks? Activate your account here