Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Issue installing Spark in windows 10.

SOLVED Go to solution

Issue installing Spark in windows 10.

New Contributor

Hi 

 

I followed below steps when installing Spark:

 

1. downlaoded JDK 10

2. Set environment varibale "JAVA_HOME" under user variable. Update the PATH under user variable to %JAVA_HOME%\bin

3. downlaod spark (spark-2.3.0-bin-hadoop2.7). Copy all files into a folder called "spark". 

4. downlaod winutils.exe from internet (GITHUB). 

5. setup environemtn varibale "HADOOP_HOME" and "SPARK_HOME"

6. Add path %SPARK_HOME%\bin

 

When i run spark-shell i am getting below error message:

C:\spark>java -version
java version "10" 2018-03-20
Java(TM) SE Runtime Environment 18.3 (build 10+46)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)

C:\spark>spark-shell
Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
        at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:791)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2464)
        at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:222)
        at org.apache.spark.deploy.SparkSubmit$.secMgr$lzycompute$1(SparkSubmit.scala:393)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$secMgr$1(SparkSubmit.scala:393)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:401)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:401)
        at scala.Option.map(Option.scala:146)
        at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:400)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:170)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end 3, length 2
        at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3107)
        at java.base/java.lang.String.substring(String.java:1873)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:52)
        ... 21 more

C:\spark>

can anyone please help with above error.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Issue installing Spark in windows 10.

New Contributor

Update: The issue is resolved by using JDK 8. 

1 REPLY 1
Highlighted

Re: Issue installing Spark in windows 10.

New Contributor

Update: The issue is resolved by using JDK 8.