https://community.cloudera.com/t5/Support-Questions/Spark-Job-long-GC-pauses/m-p/282792#M210200 <P>Hi <a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/71136">@PARTOMIA09</a>&nbsp;<BR /><BR />One suggestion off the bat is to possibly consider moving to a G1GC policy instead given that you have relatively large heap sizes (30 GB for executors and 16 GB for the driver). Typically the G1GC policy was developed to be better performant for larger heaps (&gt; 8 GB). Try the following and see if that helps:<BR /><BR /></P><LI-CODE lang="markup">--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35" --conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35"</LI-CODE><P>&nbsp;</P><P><U>"The Garbage-First Garbage Collector":</U><BR /><A href="https://www.oracle.com/technetwork/java/javase/tech/g1-intro-jsp-135488.html" target="_blank">https://www.oracle.com/technetwork/java/javase/tech/g1-intro-jsp-135488.html</A><BR /><BR /></P> Tue, 12 Nov 2019 22:42:53 GMT w@leed 2019-11-12T22:42:53Z