Agreed, this just caused me a major amount of grief as well. The "educated guessing" that is done for these locations has required manual intervention in every cluster I have configured (several dozen, across several releases/topologies).
For others reading this - several services will default to only a single partition as well, unlike HDFS which it will try to spread across multiples. In this case I saw YARN default several locations to /usr/hdp/ and an hbase tmp directory pointing to /usr/hdp/var/lib/... which is really odd. Very glad I found this thread!
... View more