It is clear you've done some research already on YARN, so I'll try to respond briefly to each to see if my responses are what you are needing to proceed. 1. In general, the RM picks a worker node that can run the AM container, so yes, good to visualize it as a round-robin approach. 2. The RM has a subtask who is responsible to watch the AM's and if one of them dies can restart it's container on another node. The AM itself needs to be written in such a way that it is restartable, but generally speaking the ones we all use (MR, Tez, Spark, etc) are all restartable. That said, "the client" may, or may not, be affected, but the "job" itself can run to completion despite an AM failure. 3. This is up to the AM to request of the RM what container sizes it needs (and how many). So, yes, theoretically they could be of different sizes, but often will be the same size. Spark is a good example as we usually ask for N containers, but want them to all have the same # of cores and amount of memory. 4. I believe the RM is going to ensure all jars are shipped to the needed NodeManager (NM) instances on the worker nodes where your containers will be at. I also believe you have some options about pre-placing your jars on HDFS, but that's been a while since I toyed with than and it was with MapReduce. Hope this helps your understanding some. Good luck!
... View more
