There have been a number of issues in the RM with relation to ZooKeeper connections. There is at least a couple of issue fixed in CDH 5.3.3 (YARN-3242, YARN-2992). I am not sure if your case is fully covered by these fixes since we are still working on one or two fixes in this area but upgrading to CDH 5.3.3 will help with a number of these ZK issues in the RM.   Wilfred ... View more

The comment on the setting in CM should have explained it for you: Maximum size in bytes for the Java Process heap memory. Passed to Java -Xmx.   You can not set that in a configuration file since the JVM is started before the configuration file is read and you need to specify the heap size on startup. CM passes the value to the agent and the agent then creates the java cmd line. The only place they are stored is in CM.   Under normal circumstances in a small cluster, lets say 10 nodes, 2 GB should be enough. In an average size cluster, 50 nodes, a RM should not need more than 4GB. In a large cluster,hundred or more NM's, you need to increase that. For a NM you can normally leave that at 1GB on small nodes or 2-4GB on large nodes. The number of containers you can run on a node is the size of the node.   Wilfred ... View more

No that is not possible. The date structure and the index number are a required part of the history path. It is hard coded in the job history server to create them and to use them while looking for the files.   Wilfred ... View more