can you provide me some hints to improve the user experience when multiple students are submitting workload on a cluster?
I have HDP 3.0 on a 5-nodes dedicated cluster (24core, 48GB each) but most times the Spark exercices on Zeppelin (directly or through Livy) stuck due to queue issues. The exercices are quite simple (not really "big data" problems) but the cumulative slowdown is a big problem for my students.
Worst, it is really hard to stress the platform only by myself, so I always finish discovering performances issues during the classes
If you have an experience tuning Zeppelin, Spark, Hive, etc., I would really appreciate your thoughts.