Hi Lester, thank you very much for your answer. I agree that there are a “TON of variables at play”. This is true that those tests are not performed on really big data and it definitely can have an impact. I have named those data sets to differ smaller and bigger data set rather than emphasising the fact that I am dealing with “really big data sets” 🙂 I will check the number of executors. Thank you for good tip. I like your logical reasoning. Even though I realize that there are a “TON” of parameters the tests were prepared reasonably precisely. The results were so surprising that I decided to post this question. Especially when everywhere I can see that Spark (which by the way I am a big fan of) is “always” faster 😉 ... View more