Reply
Explorer
Posts: 14
Registered: ‎09-04-2014

Starting Spark Service failed

Installing CDH 5.1 using CM 5.1.1. During the Cluster Setup, Starting Sark Service failed. After clicking Details, 4 child commands failed including 2 for starting this worker, 1 for starting this history server, and 1 for starting this master. In the stderr, it shows "startng spark worker using spark://hostname:7077" and then "SCALA_HOME is not set". Please help.

Explorer
Posts: 14
Registered: ‎09-04-2014

Re: Starting Spark Service failed

Mon Sep  8 19:12:42 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:42 EDT 2014
Mon Sep  8 19:12:42 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:42 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:42 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set
Mon Sep  8 19:12:43 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:43 EDT 2014
Mon Sep  8 19:12:43 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:43 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:43 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set
Mon Sep  8 19:12:45 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:45 EDT 2014
Mon Sep  8 19:12:45 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:45 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:45 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set
Mon Sep  8 19:12:49 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ echo 'Using /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/57-spark-SPARK_WORKER/scripts/control.sh start_worker ./master.properties
Mon Sep  8 19:12:49 EDT 2014
Mon Sep  8 19:12:49 EDT 2014: Detected CDH_VERSION of [5]
Mon Sep  8 19:12:49 EDT 2014: Found a master on vrdcdhm03.tactsc.com listening on port 7077
Mon Sep  8 19:12:49 EDT 2014: Starting Spark worker using spark://vrdcdhm03.tactsc.com:7077
SCALA_HOME is not set