Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Add multiple options to spark.exeuctor.extraJavaOptions

New Contributor

In HUE UI, using Spark action widget, how to set up multiple options?



-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps


This does not work in "Options List" box

--conf spark.exeuctor.extraJavaOptions="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps"



CDH 5.7.1 - Same issue, but for configuring app-specific log4j:


Working spark-submit command line: 

[--master yarn-cluster --files hdfs:/user/myao/config/ --conf "" --conf "" --class <class> <jar> ]


Cannot make the log4j portion above to work in Spark action - everything else is ok:


<action name="spark-7844">
  <spark xmlns="uri:oozie:spark-action:0.1">
    <spark-opts>--executor-memory 2G --files hdfs:/user/myao/config/ --conf &quot;; --conf &quot;; </spark-opts>
  <ok to="End"/>
  <error to="Kill"/>



Driver stderr:


Using properties file: null
Warning: Ignoring non-spark config property: ""
Warning: Ignoring non-spark config property: ""
Parsed arguments:
master yarn-cluster
deployMode cluster
executorMemory 2G
executorCores null
totalExecutorCores null
propertiesFile null
driverMemory null
driverCores null
driverExtraClassPath  ........

driverExtraLibraryPath /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/lib/native
driverExtraJavaOptions null
supervise false
queue null
numExecutors null
files hdfs:/user/myao/config/
pyFiles null
archives null
mainClass ....
primaryResource ....
name ....
childArgs []

jar  .............
packages null
packagesExclusions null
repositories null
verbose true





What version are you using ? 

Can you send the workflow.xml generated by Hue ? 

Most likely it's fixed on the latest version.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.