Support Questions
Find answers, ask questions, and share your expertise

Registry Schema failing right after starton HDF3.3

Solved Go to solution
Highlighted

Registry Schema failing right after starton HDF3.3

Explorer

I have installed a an HDF v3.3 cluster. everything seems to be working fine except Schema Registry which starts fine but then shuts down immediately giving the following errors:


The mysql drivers have been loaded fine in ambari and work for all other modules.


java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)

at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)

at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)

at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)

at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)

at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)

at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)

at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)

at io.dropwizard.cli.Cli.run(Cli.java:75)

at io.dropwizard.Application.run(Application.java:93)

at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)

Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)

... 18 more

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)

at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)

at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)

at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)

at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)

at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)

at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)

at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)

at io.dropwizard.cli.Cli.run(Cli.java:75)

at io.dropwizard.Application.run(Application.java:93)

at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)

Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)

... 18 more

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)

at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)

at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)

at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)

at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)

at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)

at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)

at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)

at io.dropwizard.cli.Cli.run(Cli.java:75)

at io.dropwizard.Application.run(Application.java:93)

at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)

Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)

... 18 more

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)

at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)

at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)

at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)

at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)

at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)

at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)

at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)

at io.dropwizard.cli.Cli.run(Cli.java:75)

at io.dropwizard.Application.run(Application.java:93)

at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)

Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)

... 18 more

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)

at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)

at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)

at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)

at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)

at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)

at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)

at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)

at io.dropwizard.cli.Cli.run(Cli.java:75)

at io.dropwizard.Application.run(Application.java:93)

at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)

Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)

... 18 more

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)

at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)

at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)

at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)

at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)

at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)

at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)

at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)

at io.dropwizard.cli.Cli.run(Cli.java:75)

at io.dropwizard.Application.run(Application.java:93)

at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)

Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)

... 18 more

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)

at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)

at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)

at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)

at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)

at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)

at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)

at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)

at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)

at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)

at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)

at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)

at io.dropwizard.cli.Cli.run(Cli.java:75)

at io.dropwizard.Application.run(Application.java:93)

at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)

Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)

... 18 more

Any ideas?

I

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Registry Schema failing right after starton HDF3.3

Explorer

Ok i was able to start shcema_registry manually by first including the mysql conenctor jar in the classpath variable:

export CLASSPATH=/usr/share/java/mysql-connector-java.jar:$CLASSPATH


To make this persistent you can enter the abo command in the "registry-env template" under the advanced config of the schema_registry service in ambari.


It would be nice to figure out and fix why this variable does not get set automatically.

View solution in original post

4 REPLIES 4
Highlighted

Re: Registry Schema failing right after starton HDF3.3

Mentor

@Lorenc Hysi

Looks like you back end database could be the cause.

################################################# 
# Create  Schema Registry and SAM metastore: 
################################################## 
mysql -u root -p{root_password} create database registry; 
create database streamline; CREATE USER 'registry'@'%' IDENTIFIED BY 'registry'; 
CREATE USER 'streamline'@'%' IDENTIFIED BY 'streamline'; 
GRANT ALL PRIVILEGES ON registry.* TO 'registry'@'%' WITH GRANT OPTION ; 
GRANT ALL PRIVILEGES ON streamline.* TO 'streamline'@'%' WITH GRANT OPTION ; 
commit; 
exit; 

################################################# 
# Test connection  
################################################## 
mysql -u streamline -pstreamline;
mysql -u registry -pregistry;


The above should succeed

It doesn't cost you a thing rerun the below to redeploy the jdbc jars

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar           
Highlighted

Re: Registry Schema failing right after starton HDF3.3

Explorer

I did all the steps above but still the same errors persists and i cannot start schema registry. I even moved the schema registry to the same server as the database but still issues.

I also tried to pick the option to let ambari create the user and database for registry during install. Ambari is able to create the user and database, but the issue still persists.


Is there any other way to force shcema registry to recognize the com.mysql.jdbc.jdbc2.optional.MysqlDataSource jar? This to me looks more like an issue with variables not being properly passed but i am not sure.

Highlighted

Re: Registry Schema failing right after starton HDF3.3

Explorer

Ok i was able to start shcema_registry manually by first including the mysql conenctor jar in the classpath variable:

export CLASSPATH=/usr/share/java/mysql-connector-java.jar:$CLASSPATH


To make this persistent you can enter the abo command in the "registry-env template" under the advanced config of the schema_registry service in ambari.


It would be nice to figure out and fix why this variable does not get set automatically.

View solution in original post

Highlighted

Re: Registry Schema failing right after starton HDF3.3

New Contributor

From what I've investigated, it may be due to the version of the mysql-connector-java.

The class com.mysql.jdbc.jdbc2.optional.MysqlDataSource is present in version 5 and the one that I have installed is the 8.

I tried installing explicitly the version 5 but I got different errors.

What I've done and got it working is changing that class to the one available in version 8, which is com.mysql.cj.jdbc.MysqlXADataSource.

 

For Schema Registry you have to modify 2 files:

  • /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-3.4.1.1-4/common-services/REGISTRY/0.3.0/package/scripts/params.py
  • /var/lib/ambari-agent/cache/common-services/REGISTRY/0.3.0/package/scripts/params.py

There change the variable registry_storage_java_class to the value com.mysql.cj.jdbc.MysqlXADataSource

Note: the variable should appear twice.

 

The same for Streaming Analytics, you have to modify 2 files. This time:

  • /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-3.4.1.1-4/common-services/STREAMLINE/0.5.0/package/scripts/params.py
  • /var/lib/ambari-agent/cache/common-services/STREAMLINE/0.5.0/package/scripts/params.py

There change the variable streamline_storage_java_class also to the value com.mysql.cj.jdbc.MysqlXADataSource

 

You should then be able to start the services.

I just solved this, so I'm not aware if any other errors will show up by using these services.

Don't have an account?