Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Solved Go to solution
Highlighted

Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Explorer

Im currently trying to Sqoop some data from an Oracle DB to HDFS using Oozie to schedule the sqoop workflow.

My Sqoop version: Sqoop 1.4.5-cdh5.4.2

My Sqoop code:

import -- connect {JDBCpath} \

--username {Username} \

--password {Password} \

--verbose \

--table {Table} \

--where "{Query}" \

-z \

--compression-codec org.apache.hadoop.io.compress.SnappyCodec \

--as-parquetfile \

--target-dir {TargetDirectory} \

--split-by {columnToSplitBy} \

-m 14

 

The Error I have been getting:

java.lang.NoSuchMethodError: org.kitesdk.data.impl.Accessor.registerDatasetRepository(Lorg/kitesdk/data/spi/URIPattern;Lorg/kitesdk/data/spi/OptionBuilder;)V

 

[Edited to focus on my main problem, rather than the research I've done into it which may or may not be the correct path in solving this. This information will be reposted in a reply.]

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Master Guru
Sqoop will use Kite if parquet-file output format is mentioned.

However, I am unable to reproduce your issue on a fresh 5.4.2 Oozie install (Sqoop action import done from a MySQL source, but Kite gets used if you specify --as-parquetfile, not dependent on the source).

Was your CDH upgraded from another release, or is it a new cluster installation?

Can you check if your Oozie system share-lib is perhaps using older kite jars? If it does look older, can you run the Oozie -> Actions -> Update ShareLib command from CM, and then retry?
5 REPLIES 5

Re: Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Master Guru
Unclear from your description, but given that CDH 5.4 includes Kite 1.0.0 in its default packaging, are you using an older version manually for some other purpose?

Re: Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Explorer

[Research I've done: It looks like the method registerDatasetRepository was included in the kite-data-core.jar file in the 0.14.1 version but was removed in version 0.15.0. We are currently using one of the kite-data-core.jar files after version 0.15.0 and I am unable to use the older version since there are other things using that jar on the hdfs.

Is there any way I can tell sqoop to not use kite or add another jar with this method included or any other solution to this error?]

 

We want to use the newest version (and as far as i know, we currently are) but the code that is generated from the sqoop command wants to use the registerDatasetRepository method from kite-data-core.jar. But it looks like that method no longer exists in the newer versions of kite-data-core.jar.

Re: Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Explorer

Would there be any way to tell oozie to not use kite when sqooping to avoid this error entirely?

Re: Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Master Guru
Sqoop will use Kite if parquet-file output format is mentioned.

However, I am unable to reproduce your issue on a fresh 5.4.2 Oozie install (Sqoop action import done from a MySQL source, but Kite gets used if you specify --as-parquetfile, not dependent on the source).

Was your CDH upgraded from another release, or is it a new cluster installation?

Can you check if your Oozie system share-lib is perhaps using older kite jars? If it does look older, can you run the Oozie -> Actions -> Update ShareLib command from CM, and then retry?

Re: Error Sqooping data with Oozie, java.lang.NoSuchMethodError

Explorer
Updating the SharLib Worked! =D I did have to bounce oozie before it fixed it but this solution works.