Created 09-18-2017 03:28 PM
I asked this question 3 times and got no valid respone , for some reason no hortonworks employee is addressing it but I am hoping someone must have done similar thing and will answer it .
I am unable to do incremental loads in ORC table.
sqoop job --create incjob -- import --connect "jdbc:oracle:thin:@(description=(address=(protocol=tcp)(host=patronQA)(port=1526))(connect_data=(service_name=patron)))" --username RON --password xxxx --incremental append --check-column txn_process_date --table PATRON.TAB1 --split-by TAB1.TXN_ID --hcatalog-database default --hcatalog-table PA_LANE_TXN_orc --create-hcatalog-table --hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY")' --map-column-hive 'txn_process_date=date ,ext_date_time=date ,ent_date_time=date' --columns 'TXN_ID,TXN_PROCESS_DATE,TRANSP_ID,TRANSP_CLASS,EXT_PLAZA_ID,EXT_LANE_ID,EXT_LANE_TYPE_CODE,EXT_DATE_TIME,TRANSP_CUR_BAL,AVC_CLASS,TOLL_AMT_CHARGED,TOLL_AMT_COLLECTED,TOLL_AMT_FULL,CREDIT_LIST_UPDATE_FLAG,TRANS_SOURCE,REVCLASS_REV_CLASS_CODE,PAYMENT_METHOD_CODE,ENT_PLAZA_ID,ENT_DATE_TIME,ENT_LANE_ID,ENT_LANE_TYPE_CODE,AGENCY_REJECT_CODE,MSG_ID,TRANSP_INTERNAL_NUM,UFM_PAYMENT_CODE,VEH_LIC_NUM,STATE_ID_CODE,ORIG_TXN_ID' Append mode for imports is not compatible with HCatalog. Please remove the parameter--append-mode [root@hadoop1 ~]#
Created 09-19-2017 12:46 AM
Created 09-19-2017 02:13 AM
As the error message says, you cannot do incremental updates using Hcatalog. See the example on this page:
Use jdbc driver to connect. Also, on the following page, read the best answer to remove any confusion you may have (there is too much redundancy on this page. Simply focus on best answer and comments on best answer).
Created 09-19-2017 04:25 AM
As I know, sqoop import "--append" options is not compatible with "--hcatalog-*". Just remove "--append".