Issue installing Spark in windows 10.

gskn — Fri, 16 Sep 2022 13:02:57 GMT

I followed below steps when installing Spark:

1. downlaoded JDK 10

2. Set environment varibale "JAVA_HOME" under user variable. Update the PATH under user variable to %JAVA_HOME%\bin

3. downlaod spark (spark-2.3.0-bin-hadoop2.7). Copy all files into a folder called "spark".

4. downlaod winutils.exe from internet (GITHUB).

5. setup environemtn varibale "HADOOP_HOME" and "SPARK_HOME"

6. Add path %SPARK_HOME%\bin

When i run spark-shell i am getting below error message:

C:\spark>java -version
java version "10" 2018-03-20
Java(TM) SE Runtime Environment 18.3 (build 10+46)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)

C:\spark>spark-shell
Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
        at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:791)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2464)
        at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:222)
        at org.apache.spark.deploy.SparkSubmit$.secMgr$lzycompute$1(SparkSubmit.scala:393)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$secMgr$1(SparkSubmit.scala:393)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:401)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:401)
        at scala.Option.map(Option.scala:146)
        at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:400)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:170)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end 3, length 2
        at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3107)
        at java.base/java.lang.String.substring(String.java:1873)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:52)
        ... 21 more

C:\spark>

can anyone please help with above error.

Re: Issue installing Spark in windows 10.

gskn — Sat, 31 Mar 2018 22:58:58 GMT

Update: The issue is resolved by using JDK 8.

question Issue installing Spark in windows 10. in Archives of Support Questions (Read Only)

Issue installing Spark in windows 10.

Re: Issue installing Spark in windows 10.