https://community.cloudera.com/t5/Support-Questions/Skew-issue-in-spark-sql/m-p/232523#M194361 <P>Hi,</P><P>I am joining two tables. One table is skewed. How to handle this in spark SQL. I am using spark 2.2.1 in AWS EMR. </P><P>Please assist on this.</P> Tue, 28 Aug 2018 18:43:57 GMT elango_rk 2018-08-28T18:43:57Z https://community.cloudera.com/t5/Support-Questions/Skew-issue-in-spark-sql/m-p/232523#M194361 <P>Hi,</P><P>I am joining two tables. One table is skewed. How to handle this in spark SQL. I am using spark 2.2.1 in AWS EMR. </P><P>Please assist on this.</P> Tue, 28 Aug 2018 18:43:57 GMT https://community.cloudera.com/t5/Support-Questions/Skew-issue-in-spark-sql/m-p/232523#M194361 elango_rk 2018-08-28T18:43:57Z https://community.cloudera.com/t5/Support-Questions/Skew-issue-in-spark-sql/m-p/232524#M194362 <P><A rel="user" href="https://community.cloudera.com/users/24259/elangork.html" nodeid="24259">@elango vaithiyanathan</A> </P><P>Perhaps you could pick another way to partition your data, by different column where the distribution of data is split evenly (hopefully)</P><P>Or else you could build an artificial (numeric) column by salting, and partition by this column.</P><P>HTH</P><P>*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.</P> Tue, 28 Aug 2018 19:10:00 GMT https://community.cloudera.com/t5/Support-Questions/Skew-issue-in-spark-sql/m-p/232524#M194362 falbani 2018-08-28T19:10:00Z https://community.cloudera.com/t5/Support-Questions/Skew-issue-in-spark-sql/m-p/232525#M194363 <P><A rel="user" href="https://community.cloudera.com/users/11048/falbani.html" nodeid="11048">@Felix Albani</A> Thank you.</P> Tue, 28 Aug 2018 19:22:21 GMT https://community.cloudera.com/t5/Support-Questions/Skew-issue-in-spark-sql/m-p/232525#M194363 elango_rk 2018-08-28T19:22:21Z