Support Questions
Find answers, ask questions, and share your expertise

is Sqoop incremental load possible for HIVE ORC table in HDP 2.5.6

Master Collaborator
if yes can you please tell me how ?

so far I am unable to get this from web searches .


Master Collaborator

anyone ? the Hortonworks staff should be able to answer this question

@Sami Ahmad

Yes, Hive incremental with append is supported in HDP 2.5.6.

Master Collaborator

but I am getting error saying its not supported

sqoop job --create incjob  -- import --connect "jdbc:oracle:thin:@(description=(address=(protocol=tcp)(host=patronQA)(port=1526))(connect_data=(service_name=patron)))" --username RON --password xxxx --incremental append --check-column txn_process_date --table PATRON.TAB1 --split-by TAB1.TXN_ID --hcatalog-database default --hcatalog-table PA_LANE_TXN_orc --create-hcatalog-table --hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY")' --map-column-hive 'txn_process_date=date ,ext_date_time=date ,ent_date_time=date' --columns 'TXN_ID,TXN_PROCESS_DATE,TRANSP_ID,TRANSP_CLASS,EXT_PLAZA_ID,EXT_LANE_ID,EXT_LANE_TYPE_CODE,EXT_DATE_TIME,TRANSP_CUR_BAL,AVC_CLASS,TOLL_AMT_CHARGED,TOLL_AMT_COLLECTED,TOLL_AMT_FULL,CREDIT_LIST_UPDATE_FLAG,TRANS_SOURCE,REVCLASS_REV_CLASS_CODE,PAYMENT_METHOD_CODE,ENT_PLAZA_ID,ENT_DATE_TIME,ENT_LANE_ID,ENT_LANE_TYPE_CODE,AGENCY_REJECT_CODE,MSG_ID,TRANSP_INTERNAL_NUM,UFM_PAYMENT_CODE,VEH_LIC_NUM,STATE_ID_CODE,ORIG_TXN_ID'

Append mode for imports is not  compatible with HCatalog. Please remove the parameter--append-mode
[root@hadoop1 ~]#
[root@hadoop1 ~]#

Master Collaborator

why are no hortonwork employees addressing this question ? I hope I am not asking some sensitive question here.

only one person replies to this question as yes its possible and then the thread dies off