Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDP3.1.0 + MariaDB: Problem deploying MariaDB connector

avatar

Hi Guys,


we are trying to install the HDP 3.1.0 platform on-premise.


We have finished the installation but ran into a problem setting up MariaDB for the Hive metastore.


Hive explicitly lists MariaDB as supported:

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+3.0+Administration

and suggests using the driver class "org.mariadb.jdbc.Driver"


When trying to setup "JDBC Driver Class" to "org.mariadb.jdbc.Driver" in Ambari of the HDP3.1.0 installation, then the start of the metastore fails with:


Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_client.py", line 60, in <module>
HiveClient().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 966, in restart
self.install(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_client.py", line 38, in install
import params
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/params.py", line 273, in <module>
else: raise Fail(format("JDBC driver '{hive_jdbc_driver}' not supported."))
resource_management.core.exceptions.Fail: JDBC driver 'org.mariadb.jdbc.Driver' not supported.


As far as I can tell, the driver class "org.mariadb.jdbc.Driver" is not included in the role out script.


Can you help us or is there a fix for this?


Thanks!

1 ACCEPTED SOLUTION

avatar
6 REPLIES 6

avatar
Master Mentor

@Maurice Knopp

Maria DB uses the MySQL drivers, you will need to the below steps to successfully deploy hive, this is my HDP 3.1 installation so I am pretty sure of what I am saying.

The Hortonworks matrix helps you choose the right versions for compatibility for HDP 3.1.0 you will need MariaDB 10.2 or MySQL 5.7 only with InnoDB

Ensure that you change the JDBC drivers as shown in the screen

# yum install -y mysql-connector-java 

Make ambari aware of the driver

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

The above setup won't destroy your Ambari installation!

108591-mariadb.png

Please let me know

avatar

We were not able to solve the problem yet. Currently, we are waiting for professional consulting support.

I will post any furhter details on that issue here when we get more information about the cause of the problem.

Thank you for your reply!

avatar
Master Mentor

@Maurice Knopp

You could save your company money 🙂 This is a trivial matter can you share your HS2 logs? What OS's are running on?

avatar

Of course we want to save money 😉

We thought the database issue is the reason we have problems connecting to the hive metastore but we now figured out that the original source for this issue seems to be related to kerberos authentication.


The main problem is, that our (new) HDP 3.1 Cluster is not running the hive scripts we developed on our "old" HDP 2.3 installation and we got lots of vertex failures.


I will try to provide more information with the help of our admin.


We are using CentOS 7.6 (all recent updates installed).

avatar
Master Mentor

@Maurice Knopp

Great news if you are on Centos and kerberized environment I am just wondering how the Kerberos have you tried regenerating the specific keytab?

About you hql scripts you should realize there has been a lot of changes between HDP 2.3 with hive 1.2.1 and HDP 3.1 with hive version 3.1.0

About your MariaDB database and running the below version with HDP 3.1

"resource_management.core.exceptions.Fail: JDBC driver 'org.mariadb.jdbc.Driver' not supported."

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 30
Server version: 5.5.60-MariaDB MariaDB Serv
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and other
Type 'help;' or '\h' for help. Type '\c' to clear the current input statemen
MariaDB [(none)]>


Please did you run the steps I mentioned in http://community.hortonworks.com/answers/245833/view.html

Please revert


avatar