Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH 5.3.1 failed to start spark master , ERROR: SCALA_HOME is not set

CDH 5.3.1 failed to start spark master , ERROR: SCALA_HOME is not set

Expert Contributor

i am working on a cluster of 3 nodes ( 2 slaves , 1 master ) and using cloudera manager 5 , i installed spark service , it starts on both workers and running , but it doesn't start on the master cause scala home not set, and below is the log :

 

Mon Mar  9 10:38:50 AST 2015
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_45-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_45-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/meta/cdh_env.sh ']'
+ OLD_IFS=' 	
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS=' 	
'
+ COUNT=1
++ seq 1 1
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.3.1-1.cdh5.3.1.p0.5
+ . /opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.3.1-1.cdh5.3.1.p0.5
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/528-spark-SPARK_MASTER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/528-spark-SPARK_MASTER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/528-spark-SPARK_MASTER/scripts/control.sh start_master ./master.properties
Mon Mar  9 10:38:50 AST 2015
Mon Mar  9 10:38:50 AST 2015: Detected CDH_VERSION of [5]
Mon Mar  9 10:38:50 AST 2015: Found a master on (private address) listening on port 7077
Mon Mar  9 10:38:50 AST 2015: Starting Spark master on (private address) and port 7077
SCALA_HOME is not set

 

how can i configure scala_home on CM ? or get this working anyway