Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎07-20-2018

sqoop command to load table data with composite key into hbase

 

 

Hi Friends, 

 

 

I am trying to use following sqoop command to move table with composite PK into hbase , but all the composite key columns are concatenated into single account column . 

 

sqoop  import --hbase-row-key 'ACCOUNT,SUBGROUP,SEGMENT,CATEGORY,' --driver com.vertica.jdbc.Driver --connect jdbc:vertica://Vertica-edw-dev.bnymellon.net:5433/vredwd01 --P  --username XBBLKDI  --table Fexchanges --hbase-table  macr01_hb:Fexchanges --column-family  account -m 1

 

how to split up these columns into respective columns 

please advise

 

Thanks 

Siva 

 

 

Explorer
Posts: 10
Registered: ‎11-07-2016

Re: sqoop command to load table data with composite key into hbase

 

Ceck whether below is helping you (add split-by in Sqoop command):

 

https://issues.cloudera.org/browse/SQOOP-107

 

Clarification:

Sqoop has no logic to handle composite primary keys.

This is for sqoop imports. If you do not specify a split-by, sqoop defaults to the PK. In the case where the table has a composite PK, it uses only the first column of the composite PK to split. This is confirmed behavior in Oracle, need to test with others.

The workaround is to specify a split-by and sqoop will not care about the PK.

 

Regards,

Nic

Announcements