Community Articles

Find and share helpful community-sourced technical articles.
Celebrating as our community reaches 100,000 members! Thank you!

Guide to deploying a 3-node DSX Local Cluster on AWS

Part 1: Deploying AWS Cluster

On your EC2 dashboard click on launch instance.

Step 1: Choose an Amazon Machine Image (AMI)

Select AWS Marketplace and search for Centos -> Chose the Centos version you need (I chose CentOS 7 (x86_64) - with Updates HVM)

Step 2: Choose an Instance Type.

Select m4.4xlarge or m4.2xlarge

Step 3: Configure Instance Details

Set Number of Instances as 3

Step 4: Add Storage

You need to add Root storage minimum 50GB and additional EBS storage of around 500GB( if you deploying for just test purpose 300GB is fine). DSX usually require high IOPS so provisioned SSD are preferred. But be advised that provisioned SSD are expensive due to high IOPS.

Step 5: Add Tags Optional

Step 6: Configure Security Group

You need to set up a custom security group, which should include -

  • Custom TCP rule, port 6443 and range -> used for internal connection
  • Allow all connection between nodes
  • Allow restricted connection from outside.

Step 7: Review and Instance Launch

You would be required to create a key-pair that will allow you to access the cluster over SSH.

Part 2: Prepare nodes to Install DSX.

After the nodes have been launched successfully, set one node as a master node. Follow the steps below to prepare the nodes for installation

Step 1. Invoking root login on each node .

  • ssh into each of the three nodes. Use the following commands -
sudo su
sudo sed -i '/^#PermitRootLogin.*/s/^#//' /etc/ssh/sshd_config
cat /etc/ssh/sshd_config | grep PermitRootLogin
  • Now, edit the ~/.ssh/authorized_keys to get rid of "no-port-forwarding,no-agent-forwarding,no-X11-forwarding,command="echo 'Please login as the user \"centos\" rather than the user \"root\".';echo;sleep 10"

Step 2. Configure password-less ssh between all the nodes

  • Creating a ssh key
  • copy content from ~/.ssh/ to ~/.ssh/authorized_keys of all nodes.
  • Test ssh connection between all the nodes including self, Note: Use the private ip of each node to ssh.

Step 3. Create two partitions for install and data.

  • confirm the creation of partitions by lsblk

Step 4. Format the two partitions

Format the partitions to mkfs.xfs -

mkfs.xfs -f -n ftype=1 -i size=512 -n size=8192 /dev/xvdb1 #//data
mkfs.xfs -f -n ftype=1 -i size=512 -n size=8192 /dev/xvdb2 #//install

Step 5. Create directories /install and /data

Step 6. Mount the new partitions onto the locations

mount /dev/xvdb1 /data
mount /dev/xvdb2 /install

Note: make following entries to /etc/fstab to ensure the partitions are mounted even on reboot.

/dev/xvdb1 /data auto defaults,nofail,comment=cloudconfig 0 2
/dev/xvdb2 /install auto defaults,nofail,comment=cloudconfig 0 2

Step 7. Install required packages

sudo yum install -y epel-release
sudo yum update -y
sudo yum install -y wget python-pip fio screen

Step 8. Improve Disk IO by warming the disks (optional)

fio --filename=/dev/xvdb --rw=read --bs=128k --iodepth=32 --ioengine=libaio --direct=1 --name=volume-initialize

Note: This will take ~40 minutes to complete

Part 3: Create Load balancers

1. Internal TCP Load balancer for Nodes

Step 1: Configure Load Balancer

Step 2: Configure Routing

Make sure the port is set to 6443.

Step 3: Register Targets register nodes in your cluster

Review and launch the load balancer. Once the ELB has been provisioned, get the IP address attached to the ELB from the DNS name - ping DNS_Name 2. Also, make sure that the target nodes has been attached to the ELB. Note: You might get warning the nodes are not healthy but its fine.

2. External Load Balancer to access cluster over HTTPS

Step 1: Configure Load Balancer

Note that here we are using port 443.

Step 2: Configure Security Settings

You can either choose your own ACM certificate or you can upload a certificate to ACM. Note: This step will work only once you have kicked off the installation. In this scenario you are creating this load balancer after the installation has been kicked off. This is good too. One way to create a ACM certificate is -

cd /etc/kubernetes/ssl

For private key use apiserver-key.pem

For certificate body use apiserver.pem

For certificate chain use ca.pem

Step 3: Configure Security Groups

Now you can configure your security group, make sure you are using same security group you earlier used to spin your instances.

Step 4: Configure Routing

Make sure to use HTTPS and set path for health check as /auth/login/login.html

Step 5: Register Targets

Review and Create the Load Balancer.

Part 4: Running DSX Installer

Step 1: Run pre-installation check

Before running installation you want to make sure that the nodes pass some basic tests. You can get the script here. Run the script, see if there are any errors while the script ran. Note: It might complain about the number of cores but you can ignore it.

Step 2: Get installer on /install folder

Download the DSX installer(optional), not needed if you have it locally. Change its permission to execute.

Step 3: Create Config file for DSX Installation

Before running the installer we would like to create a config file called as wdp.conf. This file can be created as -

your_installer --get-conf-key --three-nodes

This will create wdp.conf file. Now edit this file by replacing "load_balancer_ip_address" for "virtual_ip_address" and add the ip address of TCP load balancer. Also you would need to add 2 options to this file - Suppress Warning, No Path check. These options will suppress any warnings that you get during installation. Any errors can be later referred in the error logs.

load_balancer_ip_address=IP addr of TCP lb
node_1=internal ip address of master node
node_2=internal ip address of node2
node_3=internal ip address of node3

Step 4: Kick in the installation

Set off the installer. Note: We recommend using screen option as installation takes ~2 hr so you can restore the session if it gets disconnected from local machine.

Your_Installer --three-nodes

Part 5: Launch DSX and check the setup

After the installation has completed launch the DSX on your web browser. The ip address listed on your installer screen will not works. You will need to get the IP address of the external load balancer by ping DNS Name.


Note: You will see this warning message when you load the url, you can ignore this warning and proceed.

You should see above screen once you proceed to next step. Default username and password for first login will be - admin and password