Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Export Hive Parquet table data to Teradata: Index outof Boundary error

Highlighted

Export Hive Parquet table data to Teradata: Index outof Boundary error

Explorer

I'm trying to export data from a hdfs location to teradata. I Have created a table with same schema in teradata

 

Export Command:

sqoop export --connect jdbc:teradata://teradataserver/Database=dbname --username xxxx --password xxxx --table teradataTbl  --export-dir /hdfs/parquet/files/path/

 

Exception:

2020-03-18 14:32:00,754 ERROR [IPC Server handler 3 on 41836] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1584475869533_13501_m_000002_0 - exited : com.teradata.connector.common.exception.ConnectorException: index outof boundary
at com.teradata.connector.teradata.converter.TeradataConverter.convert(TeradataConverter.java:179)
at com.teradata.connector.common.ConnectorOutputFormat$ConnectorFileRecordWriter.write(ConnectorOutputFormat.java:111)
at com.teradata.connector.common.ConnectorOutputFormat$ConnectorFileRecordWriter.write(ConnectorOutputFormat.java:70)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:670)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at com.teradata.connector.common.ConnectorMMapper.map(ConnectorMMapper.java:134)
at com.teradata.connector.common.ConnectorMMapper.run(ConnectorMMapper.java:122)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)

 

6 REPLIES 6
Highlighted

Re: Export Hive Parquet table data to Teradata: Index outof Boundary error

Guru
@ChineduLB

Can you please share below info:

- the DDL of the table in Teradata
- some sample data from the file in HDFS

It looks to me some issues in the data, I suggest you also narrow down the issue by slowly reducing the number of columns from the file and see which one is actually causing the issue and which rows.

Cheers
Eric
Highlighted

Re: Export Hive Parquet table data to Teradata: Index outof Boundary error

Explorer

Here's the DDL

CREATE TABLE BVT_RDT.CPV_PTY_ACC_PROFL, NO FALLBACK ,
     NO BEFORE JOURNAL,
     NO AFTER JOURNAL,
     CHECKSUM = DEFAULT,
     DEFAULT MERGEBLOCKRATIO     (
      accno VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_co_no VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_appsys_id VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_sub_no VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_pd_ctgy_cd VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      aml_acc_id VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      aml_pty_id VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      accno1 VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      arng_id VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_nm VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_stat_cd_deriv VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_dom_cntry VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_opn_dt VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_cls_dt VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      cc_no VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      bnkctr_nm VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      brn_no VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_tin VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      accdm_actv_in VARCHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC,
      accpr_actv_in VARCHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acctr_actv_in VARCHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC,
      btch_id VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      lst_row_updt_ts VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_st_cd VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      jle_in VARCHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC,
      prmy_pty_in VARCHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC,
      acc_pty_role_cd VARCHAR(30) CHARACTER SET UNICODE NOT CASESPECIFIC,
      partitioncolumn VARCHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC,
      sa_data_date VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      sa_data_yyyymm VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      sa_last_updated_timestamp VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC,
      sa_last_updated_by VARCHAR(250) CHARACTER SET UNICODE NOT CASESPECIFIC      )

here's sample data

4400665121475865317555 CCA1634538008677509401000004400665121475865NULLDavidio Lynca FlavourOpenUS 2016-04-23NULLNULLNULL8BEB00140361381111201904242019-05-08 15:49:11OT10AUT12019-06-13201906132019-06-13 06:50:32.038000000qrstasluu

 

 

 

 

Highlighted

Re: Export Hive Parquet table data to Teradata: Index outof Boundary error

Guru

@ChineduLB ,

I have tested in my lab, I can successfully export the data you provided into Teradata table. Below is the output from Teradata table after Sqoop export:

accno |acc_co_no|acc_appsys_id|acc_sub_no|acc_pd_ctgy_cd|aml_acc_id|aml_pty_id|accno1 |arng_id|acc_nm |acc_stat_cd_deriv|acc_dom_cntry|acc_opn_dt|acc_cls_dt|cc_no|bnkctr_nm|brn_no|acc_tin |accdm_actv_in|accpr_actv_in|acctr_actv_in|btch_id |lst_row_updt_ts |acc_st_cd|jle_in|prmy_pty_in|acc_pty_role_cd|partitioncolumn|sa_data_date|sa_data_yyyymm|sa_last_updated_timestamp |sa_last_updated_by|
----------------|---------|-------------|----------|--------------|----------|----------|---------------------|-------|---------------------|-----------------|-------------|----------|----------|-----|---------|------|---------|-------------|-------------|-------------|--------|-------------------|---------|------|-----------|---------------|---------------|------------|--------------|-----------------------------|------------------|
4400665121475865|317 |555 | |CCA |1634538008|677509401 |000004400665121475865| |Davidio Lynca Flavour|Open |US |2016-04-23| | | |8BEB00|140361381|1 |1 |1 |20190424|2019-05-08 15:49:11|OT |1 |0 |AUT |1 |2019-06-13 |20190613 |2019-06-13 06:50:32.038000000|qrstasluu |

I think you might want to narrow down the issue by reducing the number of columns or number of records in your data to see which ones caused the issue.

Cheers
Eric

Re: Export Hive Parquet table data to Teradata: Index outof Boundary error

Explorer

Thanks EricL. At least I know it will work

Highlighted

Re: Export Hive Parquet table data to Teradata: Index outof Boundary error

Explorer

Still struggling with this... See exception stack below

 

2020-03-21 12:27:31,694 ERROR [IPC Server handler 10 on 45536] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1584785234978_6403_m_000000_0 - exited : com.teradata.connector.common.exception.ConnectorException: index outof boundary
	at com.teradata.connector.teradata.converter.TeradataConverter.convert(TeradataConverter.java:179)
	at com.teradata.connector.common.ConnectorOutputFormat$ConnectorFileRecordWriter.write(ConnectorOutputFormat.java:111)
	at com.teradata.connector.common.ConnectorOutputFormat$ConnectorFileRecordWriter.write(ConnectorOutputFormat.java:70)
	at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:670)
	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
	at com.teradata.connector.common.ConnectorMMapper.map(ConnectorMMapper.java:134)
	at com.teradata.connector.common.ConnectorMMapper.run(ConnectorMMapper.java:122)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)

 

Highlighted

Re: Export Hive Parquet table data to Teradata: Index outof Boundary error

Guru
@ChineduLB ,

Did you mean that you got the same error while trying to export the sample data you provided earlier?

Have you tried to update your driver in case it might be old?

Cheers
Eric
Don't have an account?
Coming from Hortonworks? Activate your account here