Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

KUDU need rebuild post upgrade of CDH cluster

KUDU need rebuild post upgrade of CDH cluster

Explorer

Hi Team,

 

I am facing an issue post CDH upgrade. I'm getting below error while running an application which was build on previous CDH version(5.16.1). At present I'm using CDH 6.3.3

 

Code causing error:

val kuduContext = new KuduContext(s"$KUDU_MASTER",sc);

kuduContext.upsertRows(dataframe1,s"impala::$DB.table");

 

Error which I'm getting:
20/04/01 12:26:12 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.NoSuchMethodError: org.apache.kudu.spark.kudu.KuduContext.upsertRows(Lorg/apache/spark/sql/Dataset;Ljava/lang/String;)V
java.lang.NoSuchMethodError: org.apache.kudu.spark.kudu.KuduContext.upsertRows(Lorg/apache/spark/sql/Dataset;Ljava/lang/String;)V
at com.alight.litespark.cdh_upgrade_testing_dev_1$.main(cdh_upgrade_testing_dev_1.scala:44)
at com.alight.litespark.cdh_upgrade_testing_dev_1.main(cdh_upgrade_testing_dev_1.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:673)

 

old pom.xml dependency content using which the application was built:

<dependency>
<groupId>org.apache.kudu</groupId>
<artifactId>kudu-client</artifactId>
<version>1.7.0-cdh5.16.1</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>org.apache.kudu</groupId>
<artifactId>kudu-spark2_2.11</artifactId>
<version>1.7.0-cdh5.16.1</version>
<scope>provided</scope>
</dependency>

 

After rebuilding the application with new kudu version dependency the issue got resolved.

Kindly provide the resolution so that we don't have to rebuild previous applications.

This is happening both in cluster and client mode job submission. 

Don't have an account?
Coming from Hortonworks? Activate your account here