incremental load into ORC table

[#1]

I asked this question 3 times and got no valid respone , for some reason no hortonworks employee is addressing it but I am hoping someone must have done similar thing and will answer it .

I am unable to do incremental loads in ORC table.

sqoop job --create incjob  -- import --connect "jdbc:oracle:thin:@(description=(address=(protocol=tcp)(host=patronQA)(port=1526))(connect_data=(service_name=patron)))" --username RON --password xxxx --incremental append --check-column txn_process_date --table PATRON.TAB1 --split-by TAB1.TXN_ID --hcatalog-database default --hcatalog-table PA_LANE_TXN_orc --create-hcatalog-table --hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY")' --map-column-hive 'txn_process_date=date ,ext_date_time=date ,ent_date_time=date' --columns 'TXN_ID,TXN_PROCESS_DATE,TRANSP_ID,TRANSP_CLASS,EXT_PLAZA_ID,EXT_LANE_ID,EXT_LANE_TYPE_CODE,EXT_DATE_TIME,TRANSP_CUR_BAL,AVC_CLASS,TOLL_AMT_CHARGED,TOLL_AMT_COLLECTED,TOLL_AMT_FULL,CREDIT_LIST_UPDATE_FLAG,TRANS_SOURCE,REVCLASS_REV_CLASS_CODE,PAYMENT_METHOD_CODE,ENT_PLAZA_ID,ENT_DATE_TIME,ENT_LANE_ID,ENT_LANE_TYPE_CODE,AGENCY_REJECT_CODE,MSG_ID,TRANSP_INTERNAL_NUM,UFM_PAYMENT_CODE,VEH_LIC_NUM,STATE_ID_CODE,ORIG_TXN_ID'

Append mode for imports is not  compatible with HCatalog. Please remove the parameter--append-mode
[root@hadoop1 ~]#

[#2]

Hi @Sami Ahmad,

Have you tried this? https://community.hortonworks.com/questions/58015/sqoop-hcataloghive-incremental-import-in-orc-forma...

[#3]

@Sami Ahmad

As the error message says, you cannot do incremental updates using Hcatalog. See the example on this page:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_dataintegration/content/incrementally-up...

Use jdbc driver to connect. Also, on the following page, read the best answer to remove any confusion you may have (there is too much redundancy on this page. Simply focus on best answer and comments on best answer).

https://community.hortonworks.com/questions/10710/sqoop-incremental-import-working-fine-now-i-want-k...

[#4]

As I know, sqoop import "--append" options is not compatible with "--hcatalog-*". Just remove "--append".