Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop incremental import using merge-key - Oracle BLOB data to HDFS

Sqoop incremental import using merge-key - Oracle BLOB data to HDFS

New Contributor

@ksuresh @SupportKB

I am trying to import the table with BLOB data from Oracle to HDFS using sqoop. I am getting the following error

Error: java.lang.NullPointerException at org.apache.sqoop.lib.BlobRef.writeInternal(BlobRef.java:98) at org.apache.sqoop.lib.LobRef.write(LobRef.java:307) at org.apache.sqoop.lib.LobSerializer.writeBlob(LobSerializer.java:38) at com.cloudera.sqoop.lib.LobSerializer.writeBlob(LobSerializer.java:41) at QueryResult.write(QueryResult.java:191) at org.apache.sqoop.mapreduce.MergeRecord.write(MergeRecord.java:124) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:98) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:82)

Thanks in Advance !!

5 REPLIES 5

Re: Sqoop incremental import using merge-key - Oracle BLOB data to HDFS

Hi @jyothi k

Merge key is used to pick the latest record based on primary key from old dataset and new dataset. To accomplish this the merger key should not be null. If the merge key is null then there are high chances that it will end up in null pointer exception as it will not be able to handle in sqoop import. You can use something like coalesce to replace the null value with constant and try to import it. Then it should work. Hope it helps!!

Re: Sqoop incremental import using merge-key - Oracle BLOB data to HDFS

New Contributor

Hi @Bala Vignesh N V

Actually, my table in Oracle DB has a composite primary key, but in the merge-key I am specifying only one column. So, it is throwing me the Null pointer exception.

Now, the problem is merge-key will not accept the composite primary key. So I am kind of struck there. Can you please help me if you have any idea on how to deal with this.

Thanks

Jyothi

Re: Sqoop incremental import using merge-key - Oracle BLOB data to HDFS

@jyothi k

As you said in the current version sqoop doesn't support composite keys in merger key.

So In that case you can concat both the key as concat(k1,k2) and create a uniqueId which can be used as a merge key in sqoop. By that you can overcome the limitations.

Hope it helps!

Happy Hadooping!!

Re: Sqoop incremental import using merge-key - Oracle BLOB data to HDFS

New Contributor

Hi @Bala Vignesh N V

I have another table in Oracle DB which has a single column primary key, and it also has blob data in it. When I am trying to do Sqoop import using Merge-key for this table also, it is giving me the same error - Null Pointer Exception. I think the issue is with blob data and merge-key. In the Merge-key I am passing the primary key attribute, but still it is throwing me the exception. No clue of why it is doing so.

Please help !

Thanks,

Jyothi

Re: Sqoop incremental import using merge-key - Oracle BLOB data to HDFS

@jyothi k Got it. Sqoop doesn't know to handle blob data from oracle. So you need to specify the blob to handle it in sqoop.

sqoop import --connect 'jdbc:oracle:thin:@host' --username $USER --password $Password  --table $TABLE  --hive-import --hive-table $HiveTable  --map-column-hive COL_BLOB=binary --delete-target-dir --target-dir $TargetDir  -m 1 -verbose