Support Questions

Find answers, ask questions, and share your expertise

Hive database replication with MySQL Cluster

avatar
Super Collaborator

Hi,

I would like to use replication of Hive Metastore database. To do that I installed MySQL Cluster. The problem is that MySQL Cluster needs "ndbcluster" engine for tables to replicate. My default engine is "InnoDB". After converting "hive" tables from "InnoDB" to "ndbcluster" I got strange errors (like "Expected state FINISHED, but found ERROR" or "Error while compiling statement: FAILED: SemanticException java.lang.NullPointerException") when I try to insert data to Hive. Is there any way to workaround this? Doesnt hive support "ndbcluster" engine?

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hi @Edgar Daeds,

Hive Metsatore is designed for Innodb and do not support MyISAM or ndbcluster as database type in MySQL. You can still replicate your Metastore by using MySQL replication of the binlogs.

View solution in original post

6 REPLIES 6

avatar
Super Collaborator

Hi @Edgar Daeds,

Hive Metsatore is designed for Innodb and do not support MyISAM or ndbcluster as database type in MySQL. You can still replicate your Metastore by using MySQL replication of the binlogs.

avatar
Super Collaborator

@Mats Johansson

Thank you for the answer. I did not mention that, but I want to configure High Availability for MySQL aswell. I am going to try Galera + HAProxy in the immediate future.

avatar
New Contributor

@Mats Johansson, could InnoDB Cluster be used then?

avatar
New Contributor

Hi

Any update to your work ?I am working on a similar solution with MySQL cluster and NDBcluster storage engine.

I am planning to place a VIP and a DNS in front of the MySQL cluster to act as a load balancer and single point of entry.

I too had to modify the tables and some column types to be able to import the data from InnoDB.

Regard,

Steven

avatar
Super Collaborator

I solved the problem by using MySQL master-master replication + MySQL Router as proxy. It works perfect!

avatar
Contributor

Hi Edgar,

Do you think MySQL Router is needed when I setup MariaDb galera cluster? As MariaDb galera itself takes care of active server failover.