Error Sqooping data with Oozie, java.lang.NoSuchMethodError

[#1]

Im currently trying to Sqoop some data from an Oracle DB to HDFS using Oozie to schedule the sqoop workflow.

My Sqoop version: Sqoop 1.4.5-cdh5.4.2

My Sqoop code:

import -- connect {JDBCpath} \

--username {Username} \

--password {Password} \

--verbose \

--table {Table} \

--where "{Query}" \

-z \

--compression-codec org.apache.hadoop.io.compress.SnappyCodec \

--as-parquetfile \

--target-dir {TargetDirectory} \

--split-by {columnToSplitBy} \

-m 14

The Error I have been getting:

java.lang.NoSuchMethodError: org.kitesdk.data.impl.Accessor.registerDatasetRepository(Lorg/kitesdk/data/spi/URIPattern;Lorg/kitesdk/data/spi/OptionBuilder;)V

[Edited to focus on my main problem, rather than the research I've done into it which may or may not be the correct path in solving this. This information will be reposted in a reply.]

[#2]

Unclear from your description, but given that CDH 5.4 includes Kite 1.0.0 in its default packaging, are you using an older version manually for some other purpose?

[#3]

[Research I've done: It looks like the method registerDatasetRepository was included in the kite-data-core.jar file in the 0.14.1 version but was removed in version 0.15.0. We are currently using one of the kite-data-core.jar files after version 0.15.0 and I am unable to use the older version since there are other things using that jar on the hdfs.

Is there any way I can tell sqoop to not use kite or add another jar with this method included or any other solution to this error?]

We want to use the newest version (and as far as i know, we currently are) but the code that is generated from the sqoop command wants to use the registerDatasetRepository method from kite-data-core.jar. But it looks like that method no longer exists in the newer versions of kite-data-core.jar.

[#4]

Would there be any way to tell oozie to not use kite when sqooping to avoid this error entirely?

[#5 · Accepted solution]

Sqoop will use Kite if parquet-file output format is mentioned.

However, I am unable to reproduce your issue on a fresh 5.4.2 Oozie install (Sqoop action import done from a MySQL source, but Kite gets used if you specify --as-parquetfile, not dependent on the source).

Was your CDH upgraded from another release, or is it a new cluster installation?

Can you check if your Oozie system share-lib is perhaps using older kite jars? If it does look older, can you run the Oozie -> Actions -> Update ShareLib command from CM, and then retry?

[#6]

Updating the SharLib Worked! =D I did have to bounce oozie before it fixed it but this solution works.