shc-core on HDP 25.3 using scala

[#1]

Hello,

I am trying to use spark-hbase-connector (dependency below) as part of my spark project (spark 1.6.2.2.5.3.0-37 and scala 2.10.5)

<dependency>
    <groupId>com.hortonworks</groupId>
    <artifactId>shc-core</artifactId>
    <version>1.1.1-2.1-s_2.11</version>
</dependency>

results in a noSuchMethod error:

2018-07-03 16:18:16,992 ERROR [Driver] yarn.ApplicationMaster: User class threw exception: java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
	at org.apache.spark.sql.SparkSession$Builder.config(SparkSession.scala:750)
	at org.apache.spark.sql.SparkSession$Builder.appName(SparkSession.scala:741)

Any suggestions/recommendation on what I may be overlooking would be appreciated!

Thanks,

Vinod

[#2 · Accepted solution]

@Vinod C Kaggal

You are using Spark1.x and 2.10 Scala, However the Spark and Scala version in SHC dependency are 2.1 and 2.11 respectively (1.1.1-2.1-s_2.11).

Do update the shc dependency version with "1.1.1-1.6-s_2.10" and let me know how it goes.

[#3]

@Vinod C Kaggal Please take a moment to click "Accept" if this answer helps you. 🙂

[#4]

@Sandeep Nemuri

Thanks for your response! That did help me resolve the issue.

Appreciate your help.

Vinod