I was able to solve this issue by using various combination and this is how it worked Can we make all the instances in a cluster to spot instances (for testing scenario onl) the anwser is yes In the configuration file The key think you have to remember is 1. for the instance attribute of any type of instance say master you have to have the following keyword useSpotInstances: true spotBidUSDPerHr: 2.760(This is the spot price of the instance that you are using) So the pseudo structure would be like this workers-spot {
count: 10
#
# Minimum number of instances required to set up the cluster.
# Fail and quit if minCount number of instances is not available in this cloud
# environment. Else, continue setting up the cluster.
#minCount is set to 0 always when using spot instance.
minCount: 0
instance: ${instances.d24x} {
useSpotInstances: true #required for spot instance
spotBidUSDPerHr: 2.760 #reauired for spot instance.
tags {
Name: "regionserver-REPLACE-ME"
Owner: "owner-REPLACE-ME"
}
}
roles {
HDFS: [DATANODE]
YARN: [NODEMANAGER]
HBASE: [REGIONSERVER]
}
# Optional custom role configurations
# Configuration keys containing periods must be enclosed in double quotes.
configs {
HBASE {
REGIONSERVER {
hbase_regionserver_handler_count: 64
}
}
}
}
postCreateScripts: ["""#!/bin/sh Hopefully this helps some one. Thanks Anil
... View more
