Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Starting Spark Service failed

Starting Spark Service failed

Explorer

Installing CDH 5.1 using CM 5.1.1. During the Cluster Setup, Starting Sark Service failed. After clicking Details, 4 child commands failed including 2 for starting this worker, 1 for starting this history server, and 1 for starting this master. In the stderr, it shows "startng spark worker using spark://hostname:7077" and then "SCALA_HOME is not set". Please help.

1 REPLY 1

Re: Starting Spark Service failed

Explorer

Mon Sep  8 19:12:42 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:42 EDT 2014
Mon Sep  8 19:12:42 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:42 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:42 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set
Mon Sep  8 19:12:43 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:43 EDT 2014
Mon Sep  8 19:12:43 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:43 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:43 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set
Mon Sep  8 19:12:45 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:45 EDT 2014
Mon Sep  8 19:12:45 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:45 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:45 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set
Mon Sep  8 19:12:49 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:49 EDT 2014
Mon Sep  8 19:12:49 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:49 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:49 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set