Support Questions
Find answers, ask questions, and share your expertise

Does HDP 3.1.0 support Mysql Innodb cluster

Does HDP 3.1.0 support Mysql Innodb cluster

Explorer

I'm planning to migrate my standalone MySql server containing Hive and Oozie Database to Mysql Innodb cluster for High availability.

Does HDP support it? Are there any documentation available for this set up.

6 REPLIES 6

Re: Does HDP 3.1.0 support Mysql Innodb cluster

Moderator

Hello @pauljoshiva ,

 

thank you for enquiring about how to configure Hive and Oozie services for high availability.

MySQL Inno DB engine is supported and to achieve high availability for the Oozie and Hive services, please follow this documentation.

 

I've found a good description about how to backup migrate your data to your Inno DB engine.

 

Please let me know if I've addressed your enquiry by pressing "Accept as Solution".

Should you need any further information on this topic, please do not hesitate to reach out to us.

 

Kind regards:

Ferenc


Ferenc Erdelyi, Technical Solutions Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Re: Does HDP 3.1.0 support Mysql Innodb cluster

Explorer

Hello @Bender,

    Thank you for your reply.

Your answer focuses on setting up High availability for HIVE and Oozie services but, I would like to know more on how to set up High Availability for MySql database. I understand GTID is not supported, but is there any other way apart from Master-Slave Replication method.

Re: Does HDP 3.1.0 support Mysql Innodb cluster

Moderator

Hello @pauljoshiva ,

 

thank you for the clarification that you would like to know if MySql HA is supported on HDP.

 

The short answer is: apart from the Hive Metastore, MySql HA is currently it is not supported and have not been tested. It might work, however the setup will not be supported by Cloudera.

 

There is a Community thread on this, which we did not test out, however it might be still thought provoking.

 

You are correct, GTID replication is not supported.

 

Configuring MySQL for high availability requires configuring MySQL for replication. Replication configuration depends on which version of MySQL you are using. E.g. for version 5.1 here is the documentation.

 

 

For Ambari for example If Mysql failover happens then you need to update the MySQL URL accordingly through #ambari-server setup command

  • ambari-server stop
  • backup config: cp /etc/ambari-server/conf/ambari.properties /etc/ambari
  • server/conf/ambari.properties.org
  • Run ambari-server setup
  • Enter advanced database configuration [y/n] (n)? y
  • Once setup is complete for the new MySql, start ambari-server

For other services:
If Mysql failover happens then you need to update the MySQL URL accordingly through Ambari UI for each service.
For example - hive;
Ambari ==> Hive ==> Configs ==> Advanced ==> Hive metastore
Hive Database

You can also use the Load balancer between Master and Slave nodes(MySQL HA Nodes) so that LB will redirect all incoming connections to the node which is MySQL master node. You need to change MySQL JDBC connection string like below and see if it can automatically redirect requests.

jdbc:mysql:replication://[master host][:port],[slave host 1][:port][,[slave host 2][:port]]...[/[database]]

Reference:

https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-master-slave-replication-connection.html

https://dev.mysql.com/doc/connector-j/8.0/en/

 

Hope it helps!

 

Kind regards:

Ferenc


Ferenc Erdelyi, Technical Solutions Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Re: Does HDP 3.1.0 support Mysql Innodb cluster

Explorer

Hello @Bender 

   Thanks for your reply.

Could you please let me know what kind of Load balancer I can use for MySql Master-Slave replication.

Will HAProxy work?

Re: Does HDP 3.1.0 support Mysql Innodb cluster

Moderator

Hello @pauljoshiva ,

 

my understanding is that MySQL master-slave configuration's downside is that you need to fail over manually. You can load balance between active nodes, however in my read the master-slave means that the master node is active and the slave is only follower, therefore the slave node is not serving requests, hence nothing to load balance.

 

Kind regards:

Ferenc


Ferenc Erdelyi, Technical Solutions Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Re: Does HDP 3.1.0 support Mysql Innodb cluster

Explorer

Then can I setup Master-Master Active replication with a load balancer.