Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive Table Migration (Error: java.lang.ArrayIndexOutOfBoundsException: 6) CDH 6.3.2

avatar
New Contributor

My CDH Version 6.3.2

 

I face problem that is "Error: java.lang.ArrayIndexOutOfBoundsExcpetion: 6" while migrating hive tables. But some table doesn't occur Exception as below.

The migration code has been verified to work.

I think It's a problem with upgrading the CDH version. Have you seen this error when migrating a hive table from another cluster? Another cluster CDH Version is 5.14.2 

 

1. Hadoop distcp 

2. LOAD DATA INPATH '<path>' OVERWRITE INTO TABLE <db>.<table> PARTITION (ymd=20120101)

 

Exception as below

 

```

Getting log thread is interrupted, since query is done!
Error: Error while compiling statement: FAILED: SemanticException Unable to load data to destination table. Error: java.lang.ArrayIndexOutOfBoundsException: 6 (state=42000,code=40000)
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Unable to load data to destination table. Error: java.lang.ArrayIndexOutOfBoundsException: 6
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:241)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:227)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:255)
at org.apache.hive.beeline.Commands.executeInternal(Commands.java:989)
at org.apache.hive.beeline.Commands.execute(Commands.java:1177)
at org.apache.hive.beeline.Commands.sql(Commands.java:1091)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1177)
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:840)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:898)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:518)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Unable to load data to destination table. Error: java.lang.ArrayIndexOutOfBoundsException: 6
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:329)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:207)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:260)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:505)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:491)
at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy35.executeStatementAsync(Unknown Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:295)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:507)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: Unable to load data to destination table. Error: java.lang.ArrayIndexOutOfBoundsException: 6
at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.ensureFileFormatsMatch(LoadSemanticAnalyzer.java:358)
at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:241)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:603)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1425)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1398)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:205)
... 26 more

```

 

```

Caused by: java.lang.ArrayIndexOutOfBoundsException: 6
        at org.apache.orc.OrcFile$WriterVersion.from(OrcFile.java:145)
        at org.apache.orc.impl.OrcTail.getWriterVersion(OrcTail.java:74)
        at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:385)
        at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:62)
        at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:89)
        at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1690)
        at org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:695)
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:333)
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:459)

```

 

4 REPLIES 4

avatar
Explorer

Hi Cloudera Team

 

we have upgraded cloudera 5.11 to 6.3.1 

we have orc table in hive after upgradation,not able to run the select query against in the table


Caused by: java.lang.ArrayIndexOutOfBoundsException: 6
at org.apache.orc.OrcFile$WriterVersion.from(OrcFile.java:145)
at org.apache.orc.impl.OrcTail.getWriterVersion(OrcTail.java:74)
at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:385)
at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:62)
at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:89)
at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcInputFormat.getRecordReader(VectorizedOrcInputFormat.java:186)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.createVectorizedReader(OrcInputFormat.java:1672)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1683)
at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:68)
... 16 more

 

 

 

Did you any fix for this issue

 

Regards

S.Abinanth

avatar
New Contributor

Hello,

 

We are having the exact same problem. after upgrade to CDH 6.3.2 we are unable to select on hive table. 

We understand its related to ORC version. 

Did you found a solution?

 

P.S Presto query is working on the same table...

avatar
New Contributor

Hi Team,

 

We are facing the same issue reported here while trying to fetch ORC file metadata on CDH_6.1. 

Can someone tell me what is the latest update on this? Is there any workaround available? Or any support ticket already raised?

Please let us know as this is critical for us.

 

Thanks,

Ramakrishna

avatar
Moderator

Hello @agsumeet , @direcision , @Dorown and @abinanths ,

 

thank you for reaching out! I have checked internally and the issue described in this thread looks like HIVE-16683SPARK-26932

 

As of now, there is no fix backported to any CDH release (at the time of writing it means CDH6.3.3 and earlier).

 

Should you have a Cloudera Support Subscription, please file a support case with us for further assistance!

 

Thank you:
Ferenc


Ferenc Erdelyi, Technical Solutions Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community: