I am trying to import the table with BLOB data from Oracle to HDFS using sqoop. I am getting the following error
Error: java.lang.NullPointerException at org.apache.sqoop.lib.BlobRef.writeInternal(BlobRef.java:98) at org.apache.sqoop.lib.LobRef.write(LobRef.java:307) at org.apache.sqoop.lib.LobSerializer.writeBlob(LobSerializer.java:38) at com.cloudera.sqoop.lib.LobSerializer.writeBlob(LobSerializer.java:41) at QueryResult.write(QueryResult.java:191) at org.apache.sqoop.mapreduce.MergeRecord.write(MergeRecord.java:124) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:98) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:82)
Thanks in Advance !!
Hi @jyothi k
Merge key is used to pick the latest record based on primary key from old dataset and new dataset. To accomplish this the merger key should not be null. If the merge key is null then there are high chances that it will end up in null pointer exception as it will not be able to handle in sqoop import. You can use something like coalesce to replace the null value with constant and try to import it. Then it should work. Hope it helps!!
Hi @Bala Vignesh N V
Actually, my table in Oracle DB has a composite primary key, but in the merge-key I am specifying only one column. So, it is throwing me the Null pointer exception.
Now, the problem is merge-key will not accept the composite primary key. So I am kind of struck there. Can you please help me if you have any idea on how to deal with this.
As you said in the current version sqoop doesn't support composite keys in merger key.
So In that case you can concat both the key as concat(k1,k2) and create a uniqueId which can be used as a merge key in sqoop. By that you can overcome the limitations.
Hope it helps!
Hi @Bala Vignesh N V
I have another table in Oracle DB which has a single column primary key, and it also has blob data in it. When I am trying to do Sqoop import using Merge-key for this table also, it is giving me the same error - Null Pointer Exception. I think the issue is with blob data and merge-key. In the Merge-key I am passing the primary key attribute, but still it is throwing me the exception. No clue of why it is doing so.
Please help !
@jyothi k Got it. Sqoop doesn't know to handle blob data from oracle. So you need to specify the blob to handle it in sqoop.
sqoop import --connect 'jdbc:oracle:thin:@host' --username $USER --password $Password --table $TABLE --hive-import --hive-table $HiveTable --map-column-hive COL_BLOB=binary --delete-target-dir --target-dir $TargetDir -m 1 -verbose