Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Bad Symbolic reference on SQLContext.class when compiling with spark 1.6.0-cdh5.9.1

Solved Go to solution
Highlighted

Bad Symbolic reference on SQLContext.class when compiling with spark 1.6.0-cdh5.9.1

New Contributor

Hi,

     I am using CDH 5.9.1. I submitted a spark job which uses a jar compiled with spark 1.6.0. Got following exception :

 

 

 

Exception in thread "main" java.lang.AbstractMethodError
        at org.apache.spark.Logging$class.log(Logging.scala:50)

 

So, tried to build that jar with spark 1.6.0-cdh5.9.1:

 

 

<dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_2.10</artifactId>
      <version>1.6.0-cdh5.9.1</version>
    <scope>provided</scope>
   </dependency>

 

 

It has 4 spark dependencies: spark-core, spark-catalyst, spark-hive and spark-sql. Getting following compilation error:

[ERROR] error: bad symbolic reference. A signature in SQLContext.class refers to term runtime
[INFO] in package scala.reflect which is not available.
[INFO] It may be completely missing from the current classpath, or the version on
[INFO] the classpath might be incompatible with the version used when compiling SQLContext.class.

scala-library 2.10.4 was used. Same issue was seen even changing it to 2.10.5.

 

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Bad Symbolic reference on SQLContext.class when compiling with spark 1.6.0-cdh5.9.1

New Contributor

Though compilation with spark 1.6.0-cdh5.9.1 wasn't successful, my problem got solved. I compiled my code with spark 1.6.0. That compiled successfully but while running, "java.lang.AbstractMethodError" was thrown in Spark logging api. I replaced spark logging with log4j in my code and after that it worked fine.

1 REPLY 1

Re: Bad Symbolic reference on SQLContext.class when compiling with spark 1.6.0-cdh5.9.1

New Contributor

Though compilation with spark 1.6.0-cdh5.9.1 wasn't successful, my problem got solved. I compiled my code with spark 1.6.0. That compiled successfully but while running, "java.lang.AbstractMethodError" was thrown in Spark logging api. I replaced spark logging with log4j in my code and after that it worked fine.