Created on
01-09-2023
06:10 PM
- edited on
01-10-2023
12:18 AM
by
VidyaSargur
This KB article describes how to reconfigure the DataHub cluster to use a larger VM for the master node (vertical scaling) in AWS.
Reconfiguration of the DataHub can be done in place for existing DataHubs. For new DataHubs, we recommend that you use the instance type selector from the Advanced menu to pick the desired instance type.
The DataHub will be stopped during the reconfiguration. Expect an outage that lasts for about 4 hours.
Note. Ensure that you choose an instance type that has been tested by Cloudera. For a list of appropriate instance types please consult
(Data Hub - AWS Instances list)
You change the instance type for the Master Node for the Data Hub.
The root volume size may be too small for production usage. You can resize it using the following procedure.
Use the Management Console to start the DataHub cluster
The simplest way to validate the changes is sshing into the VM(s) what you have changed and inspect the desired memory, CPU, and disk space with the appropriate Linux commands:
> free -h
total used free shared
Mem: 61G 7.3G 53G 59M
Swap: 0B 0B 0B
> lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 8
...
> df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 31G 0 31G 0% /dev
tmpfs 31G 336K 31G 1% /dev/shm
tmpfs 31G 17M 31G 1% /run
tmpfs 31G 0 31G 0% /sys/fs/cgroup
/dev/nvme0n1p1 100G 20G 81G 20% /
/dev/nvme1n1 99G 279M 94G 1% /dbfs
/dev/nvme2n1 246G 384M 234G 1% /hadoopfs/fs1
cm_processes 31G 14M 31G 1% /run/cloudera-scm-agent/process
tmpfs 6.2G 0 6.2G 0% /run/user/1001
tmpfs 6.2G 0 6.2G 0% /run/user/0