Created 03-21-2019 02:08 PM
I have installed a an HDF v3.3 cluster. everything seems to be working fine except Schema Registry which starts fine but then shuts down immediately giving the following errors:
The mysql drivers have been loaded fine in ambari and work for all other modules.
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)
at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)
at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)
at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)
at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:75)
at io.dropwizard.Application.run(Application.java:93)
at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)
... 18 more
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)
at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)
at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)
at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)
at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:75)
at io.dropwizard.Application.run(Application.java:93)
at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)
... 18 more
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)
at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)
at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)
at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)
at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:75)
at io.dropwizard.Application.run(Application.java:93)
at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)
... 18 more
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)
at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)
at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)
at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)
at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:75)
at io.dropwizard.Application.run(Application.java:93)
at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)
... 18 more
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)
at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)
at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)
at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)
at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:75)
at io.dropwizard.Application.run(Application.java:93)
at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)
... 18 more
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)
at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)
at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)
at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)
at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:75)
at io.dropwizard.Application.run(Application.java:93)
at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)
... 18 more
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:141)
at com.zaxxer.hikari.util.PoolUtilities.initializeDataSource(PoolUtilities.java:153)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:146)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.prepare(HikariCPConnectionBuilder.java:49)
at com.hortonworks.registries.storage.impl.jdbc.connection.HikariCPConnectionBuilder.<init>(HikariCPConnectionBuilder.java:38)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.getHikariCPConnnectionBuilder(QueryExecutorFactory.java:79)
at com.hortonworks.registries.storage.impl.jdbc.provider.QueryExecutorFactory.get(QueryExecutorFactory.java:45)
at com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager.init(JdbcStorageManager.java:240)
at com.hortonworks.registries.webservice.RegistryApplication.getStorageManager(RegistryApplication.java:268)
at com.hortonworks.registries.webservice.RegistryApplication.registerResources(RegistryApplication.java:184)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:80)
at com.hortonworks.registries.webservice.RegistryApplication.run(RegistryApplication.java:66)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:75)
at io.dropwizard.Application.run(Application.java:93)
at com.hortonworks.registries.webservice.RegistryApplication.main(RegistryApplication.java:297)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.zaxxer.hikari.util.PoolUtilities.createInstance(PoolUtilities.java:126)
... 18 more
Any ideas?
I
Created 03-27-2019 06:14 PM
Ok i was able to start shcema_registry manually by first including the mysql conenctor jar in the classpath variable:
export CLASSPATH=/usr/share/java/mysql-connector-java.jar:$CLASSPATH
To make this persistent you can enter the abo command in the "registry-env template" under the advanced config of the schema_registry service in ambari.
It would be nice to figure out and fix why this variable does not get set automatically.
Created 03-21-2019 10:02 PM
Looks like you back end database could be the cause.
################################################# # Create Schema Registry and SAM metastore: ################################################## mysql -u root -p{root_password} create database registry; create database streamline; CREATE USER 'registry'@'%' IDENTIFIED BY 'registry'; CREATE USER 'streamline'@'%' IDENTIFIED BY 'streamline'; GRANT ALL PRIVILEGES ON registry.* TO 'registry'@'%' WITH GRANT OPTION ; GRANT ALL PRIVILEGES ON streamline.* TO 'streamline'@'%' WITH GRANT OPTION ; commit; exit; ################################################# # Test connection ################################################## mysql -u streamline -pstreamline; mysql -u registry -pregistry;
The above should succeed
It doesn't cost you a thing rerun the below to redeploy the jdbc jars
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
Created 03-27-2019 05:53 PM
I did all the steps above but still the same errors persists and i cannot start schema registry. I even moved the schema registry to the same server as the database but still issues.
I also tried to pick the option to let ambari create the user and database for registry during install. Ambari is able to create the user and database, but the issue still persists.
Is there any other way to force shcema registry to recognize the com.mysql.jdbc.jdbc2.optional.MysqlDataSource jar? This to me looks more like an issue with variables not being properly passed but i am not sure.
Created 03-27-2019 06:14 PM
Ok i was able to start shcema_registry manually by first including the mysql conenctor jar in the classpath variable:
export CLASSPATH=/usr/share/java/mysql-connector-java.jar:$CLASSPATH
To make this persistent you can enter the abo command in the "registry-env template" under the advanced config of the schema_registry service in ambari.
It would be nice to figure out and fix why this variable does not get set automatically.
Created 10-07-2020 11:00 PM
From what I've investigated, it may be due to the version of the mysql-connector-java.
The class com.mysql.jdbc.jdbc2.optional.MysqlDataSource is present in version 5 and the one that I have installed is the 8.
I tried installing explicitly the version 5 but I got different errors.
What I've done and got it working is changing that class to the one available in version 8, which is com.mysql.cj.jdbc.MysqlXADataSource.
For Schema Registry you have to modify 2 files:
There change the variable registry_storage_java_class to the value com.mysql.cj.jdbc.MysqlXADataSource
Note: the variable should appear twice.
The same for Streaming Analytics, you have to modify 2 files. This time:
There change the variable streamline_storage_java_class also to the value com.mysql.cj.jdbc.MysqlXADataSource
You should then be able to start the services.
I just solved this, so I'm not aware if any other errors will show up by using these services.