For my case, I observed the spark job was working fine on some hosts and hitting the above exception for a couple of worker hosts.   Found that the issue with spark-submit --version on hosts. working hosts spark-submit version was version 2.4.7.7.1.7.0-551 and non-working hosts spark-submit version was version 3.1.2   I created the symbolic link with the correct spark-submit version file and the issue got resolved.   ``` [root@host bin]# cd /usr/local/bin [root@hostbin]# ln -s /etc/alternatives/spark-submit spark-submit ```     ... View more

In case it helps anyone else ....   I encountered this problem because I didn't specify any HDFS Data Nodes in my configuration.  I had specified services that depended on HDFS in the cluster in a 'master' template and had one instance of master, but my worker nodes had the HDFS: [DATANODE] role commented out. It makes sense that the cluster couldn't start.  ... View more

I experienced this issue when setting the parameter   associatePublicIpAddresses: false   The default seems to be 'true'. The notes for this parameter say ...   # Whether to associate a public IP address with instances or not. If this is false # we expect instances to be able to access the internet using a NAT instance # # Currently the only way to get optimal S3 data transfer performance is to assign # public IP addresses to your instances and not use NAT (public subnet type of setup) # # See: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-ip-addressing.html   So it makes sense that if you attempt to set this parameter to false that you may need to configure NAT as mentioned above. ... View more