Reply
Expert Contributor
Posts: 113
Registered: ‎02-15-2016

Oozie /hive metastore is not able to connect to mysql database

 

Hi ,

I have created a cluster in aws but service which connect to mysql database like oozie /hive metastore /hue are not starting since they are not able to connect  with mysql . i am getting below error , not sure what is wrong with cred files 

RROR: Oozie could not be started

REASON: org.apache.oozie.service.ServiceException: E0103: Could not load service classes, Could not load password for [oozie.service.JPAService.jdbc.password]

Stacktrace:
-----------------------------------------------------------------
org.apache.oozie.service.ServiceException: E0103: Could not load service classes, Could not load password for [oozie.service.JPAService.jdbc.password]
	at org.apache.oozie.service.Services.loadServices(Services.java:309)
	at org.apache.oozie.service.Services.init(Services.java:213)
	at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
	at org.apache.catalina.core.StandardService.start(StandardService.java:525)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:761)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.lang.IllegalArgumentException: Could not load password for [oozie.service.JPAService.jdbc.password]
	at org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:598)
	at org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:585)
	at org.apache.oozie.service.JPAService.init(JPAService.java:160)
	at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
	at org.apache.oozie.service.Services.setService(Services.java:372)
	at org.apache.oozie.service.Services.loadServices(Services.java:305)
	... 26 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:591)
	... 31 more
Caused by: java.io.IOException: Configuration problem with provider path.
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2118)
	at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2037)
	... 36 more
Caused by: java.io.IOException: Bad configuration of hadoop.security.credential.provider.path at localjceks://file/{{CMF_CONF_DIR}}/creds.localjceks
	at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:86)
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2098)
	... 37 more
Caused by: java.net.URISyntaxException: Illegal character in path at index 18: localjceks://file/{{CMF_CONF_DIR}}/creds.localjceks
	at java.net.URI$Parser.fail(URI.java:2829)
	at java.net.URI$Parser.checkChars(URI.java:3002)
	at java.net.URI$Parser.parseHierarchical(URI.java:3086)
	at java.net.URI$Parser.parse(URI.java:3034)
	at java.net.URI.<init>(URI.java:595)
	at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:67)
	... 38 more
-----------------------------------------------------------------

 

 

 

Cloudera Employee
Posts: 420
Registered: ‎03-23-2015

Re: Oozie /hive metastore is not able to connect to mysql database

This looks like that your *-site.xml files are invalid. Can you check the content of oozie-site.xml file and confirm database settings are correct?
Expert Contributor
Posts: 113
Registered: ‎02-15-2016

Re: Oozie /hive metastore is not able to connect to mysql database

database settings are correct , even i removed the oozie and added it back with new database /username/password  and it was able to connect with mysql but its failing at time of creation of tables in mysql  with error 

 

Wed May 23 14:11:22 EDT 2018
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
using /var/lib/oozie/tomcat-deployment as CATALINA_BASE
CONF_DIR=/run/cloudera-scm-agent/process/196-oozie-OOZIE_SERVER
CMF_CONF_DIR=/etc/cloudera-scm-agent
Copying JDBC jar from /usr/share/java/mysql-connector-java.jar to /var/lib/oozie

ERROR: Oozie could not be started

REASON: org.apache.oozie.service.ServiceException: E0103: Could not load service classes, Could not load password for [oozie.service.JPAService.jdbc.password]

Stacktrace:
-----------------------------------------------------------------
org.apache.oozie.service.ServiceException: E0103: Could not load service classes, Could not load password for [oozie.service.JPAService.jdbc.password]
	at org.apache.oozie.service.Services.loadServices(Services.java:309)
	at org.apache.oozie.service.Services.init(Services.java:213)
	at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
	at org.apache.catalina.core.StandardService.start(StandardService.java:525)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:761)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.lang.IllegalArgumentException: Could not load password for [oozie.service.JPAService.jdbc.password]
	at org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:598)
	at org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:585)
	at org.apache.oozie.service.JPAService.init(JPAService.java:160)
	at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
	at org.apache.oozie.service.Services.setService(Services.java:372)
	at org.apache.oozie.service.Services.loadServices(Services.java:305)
	... 26 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:591)
	... 31 more
Caused by: java.io.IOException: Configuration problem with provider path.
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2118)
	at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2037)
	... 36 more
Caused by: java.io.IOException: Bad configuration of hadoop.security.credential.provider.path at localjceks://file/{{CMF_CONF_DIR}}/creds.localjceks
	at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:86)
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2098)
	... 37 more
Caused by: java.net.URISyntaxException: Illegal character in path at index 18: localjceks://file/{{CMF_CONF_DIR}}/creds.localjceks
	at java.net.URI$Parser.fail(URI.java:2829)
	at java.net.URI$Parser.checkChars(URI.java:3002)
	at java.net.URI$Parser.parseHierarchical(URI.java:3086)
	at java.net.URI$Parser.parse(URI.java:3034)
	at java.net.URI.<init>(URI.java:595)
	at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:67)
	... 38 more
-----------------------------------------------------------------
Announcements