@Jona  Great article...may i ask can you use "custom spark2-metrics-properties" in spark config instead of advanced spark2-metrics-properties? ... View more

what is the exact solution? "I solved this issue by using spark 2.3.1 jars under /usr/hdp/current/spark2-client/ from the HDP3.0 cluster." I get this on spark-submit. Are you saying that i should pass --jars /usr/hdp/current/spark2-client/*.jar ? ... View more

what is the exact solution? "I solved this issue by using spark 2.3.1 jars under /usr/hdp/current/spark2-client/ from the HDP3.0 cluster." I get this on spark-submit. Are you saying that i should pass --jars /usr/hdp/current/spark2-client/*.jar ? ... View more