Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDF SAM JDBC sink

Highlighted

HDF SAM JDBC sink

New Contributor

Hi All I'm new here.

I'm developing simple stream with JDBC sink to mysql database.
How to register MYSQL(or any other JDBC ) driver in a way that it would be accessible for storm?
I'm getting errors that my driver class name can't be found by Hikari.

*I have installed mysql client and JDBC drivers on all nodes in my cluster.

BR
Grzegorz

3 REPLIES 3
Highlighted

Re: HDF SAM JDBC sink

Contributor

@gbuchcik Are they errors in storm worker logs? Can you please paste them here? One thing you can try is change your worker.childopts in ambari(if you are using it) to make sure its passed in your classpath so that its available to the worker jvm process.

Highlighted

Re: HDF SAM JDBC sink

New Contributor

yes those are errors from storm worker, below you can find example.
I do not use worker.childopts in ambari but I can experiment to change its values ("TOPOLOGY WORKER JVM OPTIONS") while deploying topology from SAM. What do you suggest to put there ?


java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.zaxxer.hikari.util.PropertyElf.setProperty(PropertyElf.java:171) at com.zaxxer.hikari.util.PropertyElf.setTargetFromProperties(PropertyElf.java:62) at com.zaxxer.hikari.HikariConfig.<init>(HikariConfig.java:128) at org.apache.storm.jdbc.common.HikariCPConnectionProvider.prepare(HikariCPConnectionProvider.java:46) at org.apache.storm.jdbc.bolt.AbstractJdbcBolt.prepare(AbstractJdbcBolt.java:49) at org.apache.storm.jdbc.bolt.JdbcInsertBolt.prepare(JdbcInsertBolt.java:78) at com.hortonworks.streamline.streams.runtime.storm.bolt.jdbc.StreamlineJdbcInsertBolt.prepare(StreamlineJdbcInsertBolt.java:20) at org.apache.storm.daemon.executor$fn__9866$fn__9879.invoke(executor.clj:794) at org.apache.storm.util$async_loop$fn__555.invoke(util.clj:482) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.zaxxer.hikari.util.PropertyElf.setProperty(PropertyElf.java:163) ... 10 more Caused by: java.lang.RuntimeException: Could not load class of driverClassName com.mysql.jdbc.Driver at com.zaxxer.hikari.HikariConfig.setDriverClassName(HikariConfig.java:330) ... 15 more Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at com.zaxxer.hikari.HikariConfig.setDriverClassName(HikariConfig.java:325)

Re: HDF SAM JDBC sink

I have the same issue, trying to connect to a Postgres remote database. I guess the real question is: Where do I put the JDBC jar and how do I specify it in the "Driver Class Name" in the JDBC Sink. Currently, the jar file is located at /usr/hdf/3.1.2.0-7/streamline/libs/postgresql-9.4.1212.jar and the Driver Class Name that I specified is org.postgresql.postgresql.postgresql-9.4.1212. This is obviously wrong, but what is right?

The following Driver Class Names did not work:

  • org.postgresql.Driver
  • /usr/hdf/3.1.2.0-7/streamline/libs/postgresql-9.4.1212 (with or without ".jar" extension)
  • postgresql-9.4.1212.jar
  • org.postgresql.postgresql.postgresql-9.4.1212.jar
  • libs.postgresql-9.4.1212.jar
Don't have an account?
Coming from Hortonworks? Activate your account here