Spark: avoiding stack traces when starting spark-shell

[#1]

When starting spark-shell, it tries to bind to port 4040 for the SparkUI.

If that port is already taken because of another spark-shell session active, it tries then to bind on 4041, then 4042, etc.

Each time the binding didn't suceed, there's a huge WARN stack trace which could be filtered

[user@serv hive]$ SPARK_MAJOR_VERSION=2 spark-shell 
SPARK_MAJOR_VERSION is set to 2, using Spark2 
Setting default log level to "WARN". 
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 
17/09/20 11:49:43 WARN AbstractLifeCycle: FAILED ServerConnector@2d258eff{HTTP/1.1}
{0.0.0.0:4040}: java.net.BindException: Address already in use 
java.net.BindException: Address already in use 
at sun.nio.ch.Net.bind0(Native Method) 
at sun.nio.ch.Net.bind(Net.java:433) 
at sun.nio.ch.Net.bind(Net.java:425) 
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) 
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) 
at org.spark_project.jetty.server.ServerConnector.open(ServerConnector.java:321) 
at org.spark_project.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) 
at org.spark_project.jetty.server.ServerConnector.doStart(ServerConnector.java:236) 
at org.spark_project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) 
at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$newConnector$1(JettyUtils.scala:333)
at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$httpConnect$1(JettyUtils.scala:365) 
at org.apache.spark.ui.JettyUtils$$anonfun$7.apply(JettyUtils.scala:368) 
at org.apache.spark.ui.JettyUtils 

To filter that stacktrace, let's put that class log4j verbosity in ERROR level in /usr/hdp/current/spark2-client/conf/log4j.properties

# Added for not having stack traces when binding to SparkUI
log4j.logger.org.spark_project.jetty.util.component.AbstractLifeCycle=ERROR