Is there any efficient way of autoscaling up/down of the clusters?
I understand scaling up will be easier as we are just adding the resources which makes the work easier. But when the nodes are idle and when we think about scaling down as there is no need for more resources , then comes the hard part. 1. We need to think about the current jobs which are running
2.If the nodes are brought down then the jobs running on the nodes will also be taken into consideration.
So without affecting the current running jobs / reducing the current running jobs, is there a way to scale down the cluster.
Note: Scaling up/down is for cloud based environment when it comes to on-premises scaling up/down is completely out of the picture in short time.