https://community.cloudera.com/t5/Support-Questions/Spark-can-t-join-dataframes-without-using-over-a-hundred-GB/m-p/222532#M184402 <P><A rel="user" href="https://community.cloudera.com/users/83321/robcornell.html" nodeid="83321">@Robert Cornell</A><BR />We have a similar issue with joining, even after bucketing and presorting of both tables it still throws to us this kind of behavior(in theory it simply should zip both dataframes without any additional operations). </P><P><BR />Only one difference is we use Spark SQL and outer join instead of RDD, but the symptoms look quite similar. Did you manage to fix it?</P> Wed, 10 Oct 2018 18:02:03 GMT ssemyonov 2018-10-10T18:02:03Z