question spark job shuffle write super slow in Support Questions

question spark job shuffle write super slow in Support Questions https://community.cloudera.com/t5/Support-Questions/spark-job-shuffle-write-super-slow/m-p/220400#M182285 <img src="https://community.cloudera.com/t5/image/serverpage/image-id/15886iB716AC251285499A/image-size/medium?v=v2&px=400" role="button" title="76637-untitled2.png" alt="76637-untitled2.png" /><img src="https://community.cloudera.com/t5/image/serverpage/image-id/15887iD5BC8D822193D1F6/image-size/medium?v=v2&px=400" role="button" title="76646-untitled2.png" alt="76646-untitled2.png" />why is the spark shuffle stage is so slow for 1.6 MB shuffle write, and 2.4 MB input?.Also why is the shuffle write happening only on one executor ?.I am running a 3 node cluster with 8 cores each.Please see my code and Spark UI pictures belowCode:<PRE><code>JavaPairRDD<String, String> javaPairRDD = c.mapToPair(new PairFunction<String, String, String>() { @Override public Tuple2<String, String> call(String arg0) throws Exception { // TODO Auto-generated method stub try { if (org.apache.commons.lang.StringUtils.isEmpty(arg0)) { return new Tuple2<String, String>("", ""); } Tuple2<String, String> t = new Tuple2<String, String>(getESIndexName(arg0), arg0); return t; } catch (Exception e) { e.printStackTrace(); System.out.println("******* exception in getESIndexName"); } return new Tuple2<String, String>("", ""); } }); java.util.Map<String, Iterable<String>> map1 = javaPairRDD.groupByKey().collectAsMap();* </PRE><img src="https://community.cloudera.com/t5/image/serverpage/image-id/15888iCE03E57EF3577E96/image-size/medium?v=v2&px=400" role="button" title="76636-untitled1.png" alt="76636-untitled1.png" /> Sun, 18 Aug 2019 02:28:41 GMT pradeepbill 2019-08-18T02:28:41Z