Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive database replication with MySQL Cluster

Solved Go to solution
Highlighted

Hive database replication with MySQL Cluster

Expert Contributor

Hi,

I would like to use replication of Hive Metastore database. To do that I installed MySQL Cluster. The problem is that MySQL Cluster needs "ndbcluster" engine for tables to replicate. My default engine is "InnoDB". After converting "hive" tables from "InnoDB" to "ndbcluster" I got strange errors (like "Expected state FINISHED, but found ERROR" or "Error while compiling statement: FAILED: SemanticException java.lang.NullPointerException") when I try to insert data to Hive. Is there any way to workaround this? Doesnt hive support "ndbcluster" engine?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hive database replication with MySQL Cluster

Expert Contributor

Hi @Edgar Daeds,

Hive Metsatore is designed for Innodb and do not support MyISAM or ndbcluster as database type in MySQL. You can still replicate your Metastore by using MySQL replication of the binlogs.

View solution in original post

6 REPLIES 6
Highlighted

Re: Hive database replication with MySQL Cluster

Expert Contributor

Hi @Edgar Daeds,

Hive Metsatore is designed for Innodb and do not support MyISAM or ndbcluster as database type in MySQL. You can still replicate your Metastore by using MySQL replication of the binlogs.

View solution in original post

Highlighted

Re: Hive database replication with MySQL Cluster

Expert Contributor

@Mats Johansson

Thank you for the answer. I did not mention that, but I want to configure High Availability for MySQL aswell. I am going to try Galera + HAProxy in the immediate future.

Highlighted

Re: Hive database replication with MySQL Cluster

New Contributor

@Mats Johansson, could InnoDB Cluster be used then?

Highlighted

Re: Hive database replication with MySQL Cluster

New Contributor

Hi

Any update to your work ?I am working on a similar solution with MySQL cluster and NDBcluster storage engine.

I am planning to place a VIP and a DNS in front of the MySQL cluster to act as a load balancer and single point of entry.

I too had to modify the tables and some column types to be able to import the data from InnoDB.

Regard,

Steven

Highlighted

Re: Hive database replication with MySQL Cluster

Expert Contributor

I solved the problem by using MySQL master-master replication + MySQL Router as proxy. It works perfect!

Highlighted

Re: Hive database replication with MySQL Cluster

Hi Edgar,

Do you think MySQL Router is needed when I setup MariaDb galera cluster? As MariaDb galera itself takes care of active server failover.

Don't have an account?
Coming from Hortonworks? Activate your account here