in anyway you should go with 3 node zk quorum with 3 nodes ensemble, that means 2 servers must be up at any time. the more members an ensemble has, the more tolerant the ensemble is of host failures.
... View more
if worker goes down then supervisor will try to restart it, unable to do so after trying multiple times Nimbus will reassign tasks to another worker slots.Also in the event of worker node failure Nimbus will reassign the tasks on other nodes.
... View more
@Srinivas Badveli did you try HADOOP_USER_CLASSPATH_FIRST (mapreduce.job.user.classpath.first) which actually allow prepend user app classpath first so the desired lib can be picked up.
... View more
please refer http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/de//archive/mapreduce-osdi04.pdf most importantly the implementation
... View more
you can update this in conf/spark-env.sh file and also overwrite it with your job options like this --conf spark.storage.memoryFraction=0.4
... View more
