Created 10-23-2017 07:19 PM
I am trying to import from oracle to hive. without --map-column-hive 'ae_header_id=bigint,application_id=smallint,entity_id=bigint,event_id=bigint,accounting_date=date,gl_transfer_date=timestamp' \ its loading fine but getting error only with map-column-hive.
source and destination tables are having same column names.
Created 10-23-2017 11:29 PM
SQOOP Script:
--query 'select ae_header_id ,
application_id ,
entity_id ,
event_id ,
event_type_code ,
accounting_date ,
gl_transfer_status_code ,
gl_transfer_date
from xxcfi_xla_ae_head_test where 1=1 and $CONDITIONS' --split-by ae_header_id --m 10 \
--hive-import \
--hive-table xxcfi.xxcfi_xla_ae_head_new6 \
--map-column-hive 'ae_header_id=bigint,application_id=smallint,entity_id=bigint,event_id=bigint,accounting_date=date,gl_transfer_date=timestamp' \
Oracle Table def for these columns:
AE_HEADER_ID NUMBER(15),
APPLICATION_ID NUMBER(15),
LEDGER_ID NUMBER(15),
ENTITY_ID NUMBER(15),
EVENT_ID NUMBER(15),
EVENT_TYPE_CODE VARCHAR2(30 BYTE),
ACCOUNTING_DATE DATE,
GL_TRANSFER_STATUS_CODE VARCHAR2(30 BYTE),
GL_TRANSFER_DATE DATE,
HIve table:
ae_header_id BIGINT,
application_id SMALLINT,
entity_id BIGINT,
event_id BIGINT,
event_type_code string,
accounting_date date,
gl_transfer_status_code string,
gl_transfer_date timestamp
HIve:
CREATE TABLE xxcfi_xla_ae_head_new6 (
ae_header_id BIGINT,
application_id SMALLINT,
entity_id BIGINT,
event_id BIGINT,
event_type_code string,
accounting_date date,
gl_transfer_status_code string,
gl_transfer_date timestamp,
)
-- PARTITIONED BY(ledger_id INT)
CLUSTERED BY (ae_header_id) sorted BY (accounting_date) INTO 500 BUCKETS
STORED AS TEXTFILE
LOCATION '/user//hive/warehouse/xxcfi_xla_ae_head_new6';
Created 10-29-2017 06:02 AM
@Kishore Jasthi Can you try using '--hcatalog-table' instead of '--hive-table'?