Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

shc-core on HDP 25.3 using scala

avatar
New Contributor

Hello,

I am trying to use spark-hbase-connector (dependency below) as part of my spark project (spark 1.6.2.2.5.3.0-37 and scala 2.10.5)

<dependency>
<groupId>com.hortonworks</groupId>
<artifactId>shc-core</artifactId>
<version>1.1.1-2.1-s_2.11</version>
</dependency>

results in a noSuchMethod error:

2018-07-03 16:18:16,992 ERROR [Driver] yarn.ApplicationMaster: User class threw exception: java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
	at org.apache.spark.sql.SparkSession$Builder.config(SparkSession.scala:750)
	at org.apache.spark.sql.SparkSession$Builder.appName(SparkSession.scala:741)

Any suggestions/recommendation on what I may be overlooking would be appreciated!

Thanks,

Vinod

1 ACCEPTED SOLUTION

avatar
@Vinod C Kaggal

You are using Spark1.x and 2.10 Scala, However the Spark and Scala version in SHC dependency are 2.1 and 2.11 respectively (1.1.1-2.1-s_2.11).

Do update the shc dependency version with "1.1.1-1.6-s_2.10" and let me know how it goes.

View solution in original post

3 REPLIES 3

avatar
@Vinod C Kaggal

You are using Spark1.x and 2.10 Scala, However the Spark and Scala version in SHC dependency are 2.1 and 2.11 respectively (1.1.1-2.1-s_2.11).

Do update the shc dependency version with "1.1.1-1.6-s_2.10" and let me know how it goes.

avatar

@Vinod C Kaggal Please take a moment to click "Accept" if this answer helps you. 🙂

avatar
New Contributor

@Sandeep Nemuri

Thanks for your response! That did help me resolve the issue.

Appreciate your help.

Vinod