MySQL Cluster on HDF 3.1.0

[#1]

Hi,

We are planning to deploy HDF 3.1.0 sandbox to two VMs to have clustered NiFi and we have the following questions:

1. Does the HDF sandbox contains MySQL as part of the sandbox.

2. Can we have our data node (MySQL DB) inside the smae VM which has the HDF.

2. Does HDF docker image can be deployed to Clustered Linux VMs.

3. If we deployed two HDF to two VMs and we manage to create NiFI cluster, can we have MySQL cluster in this case.

4. What is the recommended VM Specs in term of RAM, CPU.

Thanks.

Regards,

[#2]

It sounds like you want to create a NiFi cluster - rather than using two separate sandboxes on two separate VMs, I would use our HDF installation guide to set up a full cluster with Ambari and two NiFi nodes. Documentation for that can be found here:

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_installing-hdf/content/ch_install-ambari...

The Sandbox is meant as exactly the name implies - a sandbox for understanding and testing out the various technologies in the stack. It sounds like you are starting to approach a more advanced set of scenarios and use cases (though I could be wrong), so I would recommend setting up a cluster.

To answer your other questions:
2. Yes, you can have MySQL installed on the same server as a HDF node is installed

3. I'm not sure what you mean by this question, could you rephrase or elaborate? You can create a MySQL HA configuration but this is outside the scope of HDF cluster setup and you would be doing this independent of the Ambari UI setup steps. You can use the nodes in your HDF cluster to setup MySQL HA but you should consider organizational SLA requirements and up-time to understand whether to co-locate or to separate HDF and MySQL

4. Recommended VM specs for a 3-node NiFi cluster are at least 8GB RAM, 8 cores, and 8 disks for a production setup. More details here:

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_planning-your-deployment/content/ch_hard...