we have a brainstorm about the best way to implement solutions for our stream and processing layer (including ad-hoc reports) and our DWH cluster .
the question is between 2 approaches : a unified cluster over HDFS , that serves the steam app over yarn , and the DWH needs over kubernetes (it is a working and closed solution so it's given that it will use kubernetes).
note that the app A (stream app) will have completely other SLA and Qos than the app B (DWH )
the other approach is to have 2 separate clusters that will potentially demand more managing .
so the questions are :
1. is it feasible to accommodate both needs on the same cluster, with logic border between the two , for different resource utilization and Qos ?
2. if both apps will use spark , can spark use kubernetes for app B and yarn for app A ?
3. is their an known example of such a division in the production ?