Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

create table as select failed, but only when stored as parquet

avatar
Explorer

Hello,

 

A simple create table as select fails for some tables in my original_db and I don't know why.

The query I use is below, and the error HIVE gives out is really long, but the main theme is "Vertex Failed", "Error while processing row", "No enum constant" - as you can see below.

 

However, when ran without "STORED AS PARQUET" option, the query runs fine.

Any ideas what I can try to maybe create the tables as compressed parquet?

 

Thanks

 

 

 

beeline -e "SET parquet.compression=SNAPPY;
SET hive.exec.compress.output=true;
SET parquet.output.codec=snappy;
set hive.tez.container.size = 8192;


DROP table if exists DB.table;
CREATE table DB.table
STORED AS parquet

As


SELECT * FROM ORIGINAL_DB.table;"

 

ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1640013186854_27946_1_00, diagnostics=[Task failed, taskId=task_1640013186854_27946_1_00_000004, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1640013186854_27946_1_00_000004_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

[...]

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5

 

1 ACCEPTED SOLUTION

avatar
Contributor

@vladenache The issue seems to be with field 

FieldSchema(name:80t_lab.fan_glo (q5), type:tinyint, comment:null)], properties:null)

Please check schema of the table for above field and correct it's name. From the attached stack trace it is trying to identify q5 as a data type and failing as it does not exist 

 

View solution in original post

6 REPLIES 6

avatar
Contributor

Hi @vladenache 

 

From the pasted stack trace, we could see that enum q5 does not exist. From the code base 

org.apache.parquet.schema.OriginalType

please refer to https://github.com/apache/parquet-mr/blob/master/parquet-column/src/main/java/org/apache/parquet/sch...

So the problem could be with 

  1. Incompatible versions of Parquet/Hive in use, please let us know what versions of hive, parquet are in use or Hadoop distribution version also helps
  2. Source data format issues, please share schema of source table and sample data would help
  3. Please share full stack trace to understand more about code execution path while query is getting executed

avatar
Explorer

Hello @tarak271 , thanks for your answer 

 

Here are some more details

- Hadoop 3.1.1.7.1.7.63-1

[ I can't manage to get to the parquet version currently in use ]

- the table comes initially from a sqoop job, so maybe some invalid data slipped thru. over 100 columns, so difficult to present them here ( can't attach a sample in my response ) - only 3 types : int, string, double

 

I thought also it might be an invalid value in one of the columns but because I am not returned a row number along with the error message it is impossible to see where that would be.

 

 

 

 

avatar
Contributor

HI @vladenache 

 

Could you please share next 50 lines after the below line

 

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5

 

or full output of beeline console which includes full stack trace

avatar
Explorer

It appears several times in the log

 

Here are the first lines of the error, which include that exception also:

 

ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1640013186854_27946_1_00, diagnostics=[Task failed, taskId=task_1640013186854_27946_1_00_000004, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1640013186854_27946_1_00_000004_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:437)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
	... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
	... 19 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:829)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:1004)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:937)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:937)
	at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
	at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
	... 20 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5
	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:282)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketForFileIdx(FileSinkOperator.java:872)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:823)
	... 27 more
Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5
	at java.lang.Enum.valueOf(Enum.java:238)
	at org.apache.parquet.schema.OriginalType.valueOf(OriginalType.java:24)
	at org.apache.parquet.schema.MessageTypeParser.addPrimitiveType(MessageTypeParser.java:182)
	at org.apache.parquet.schema.MessageTypeParser.addType(MessageTypeParser.java:113)
	at org.apache.parquet.schema.MessageTypeParser.addGroupTypeFields(MessageTypeParser.java:101)
	at org.apache.parquet.schema.MessageTypeParser.parse(MessageTypeParser.java:94)
	at org.apache.parquet.schema.MessageTypeParser.parseMessageType(MessageTypeParser.java:84)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.getSchema(DataWritableWriteSupport.java:51)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.init(DataWritableWriteSupport.java:57)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:418)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:380)
	at org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.<init>(ParquetRecordWriterWrapper.java:70)
	at org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getParquerRecordWriterWrapper(MapredParquetOutputFormat.java:137)
	at org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getHiveRecordWriter(MapredParquetOutputFormat.java:126)
	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter(HiveFileFormatUtils.java:294)
	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:279)
	... 29 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1640013186854_27946_1_00_000004_1:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:437)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
	... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
	... 19 more

 

Also, here is the schema created

 

INFO  : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:80t_lab.time, type:string, comment:null), FieldSchema(name:80t_lab.trigger, type:int, comment:null), FieldSchema(name:80t_lab.singlstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.temppi_usecfc, type:double, comment:null), FieldSchema(name:80t_lab.humipi_usecfd, type:double, comment:null), FieldSchema(name:80t_lab.locorshtempglo, type:double, comment:null), FieldSchema(name:80t_lab.supply_temp_unfiltered, type:double, comment:null), FieldSchema(name:80t_lab.temp_setpoint, type:double, comment:null), FieldSchema(name:80t_lab.fan_setpoint, type:double, comment:null), FieldSchema(name:80t_lab.temp_local, type:double, comment:null), FieldSchema(name:80t_lab.return_temp, type:double, comment:null), FieldSchema(name:80t_lab.ht_humi, type:double, comment:null), FieldSchema(name:80t_lab.ht_dewpcorr, type:double, comment:null), FieldSchema(name:80t_lab.fc_temp, type:double, comment:null), FieldSchema(name:80t_lab.bara_fan, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_cool, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_deh, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_heate, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_hum, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_fc, type:tinyint, comment:null), FieldSchema(name:80t_lab.c1a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c1b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c2a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c2b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c3a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c3b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c4a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c4b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp1, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp2, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp3, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp4, type:tinyint, comment:null), FieldSchema(name:80t_lab.sys_pi_temp, type:double, comment:null), FieldSchema(name:80t_lab.sys_pi_fan, type:double, comment:null), FieldSchema(name:80t_lab.sys_pi_humi, type:double, comment:null), FieldSchema(name:80t_lab.c1_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c2_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c3_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c4_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c1_disct, type:double, comment:null), FieldSchema(name:80t_lab.c2_disct, type:double, comment:null), FieldSchema(name:80t_lab.c3_disct, type:double, comment:null), FieldSchema(name:80t_lab.c4_disct, type:double, comment:null), FieldSchema(name:80t_lab.slp1_status, type:string, comment:null), FieldSchema(name:80t_lab.slp2_status, type:string, comment:null), FieldSchema(name:80t_lab.slp3_status, type:string, comment:null), FieldSchema(name:80t_lab.slp4_status, type:string, comment:null), FieldSchema(name:80t_lab.pb1_status, type:string, comment:null), FieldSchema(name:80t_lab.pb2_status, type:string, comment:null), FieldSchema(name:80t_lab.pb3_status, type:string, comment:null), FieldSchema(name:80t_lab.pb4_status, type:string, comment:null), FieldSchema(name:80t_lab.pb1opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pb2opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pb3opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pb4opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc1sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc2sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc3sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc4sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.leadlag_swap, type:string, comment:null), FieldSchema(name:80t_lab.dxqs_status_glo, type:string, comment:null), FieldSchema(name:80t_lab.pb1_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pb2_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pb3_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pb4_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pre1_pump_request_mode, type:string, comment:null), FieldSchema(name:80t_lab.pre1_pumpstartupstatus, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre1_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre2_pump_request_mode, type:string, comment:null), FieldSchema(name:80t_lab.pre2_pumpstartupstatus, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre2_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre3_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre4_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre_early_cmprtomixed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre_early_mixedtopump, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre_early_cmprtopump, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_tempamb, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_cntrlstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb11_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb12_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb11_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb12_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan1actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan2actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan3actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan1spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan2spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan3spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcc1_temprfrg, type:double, comment:null), FieldSchema(name:80t_lab.gcc1_tempcondset, type:double, comment:null), FieldSchema(name:80t_lab.gcc1_prescond, type:double, comment:null), FieldSchema(name:80t_lab.gcc1_prescondset, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_temprefrig, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_prescond, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_prescondset, type:double, comment:null), FieldSchema(name:80t_lab.gcb2_tempamb, type:double, comment:null), FieldSchema(name:80t_lab.gcb2_cntrlstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb21_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb22_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb21_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb22_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb2_fan1actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb2_fan1spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcc2_temprfrg, type:double, comment:null), FieldSchema(name:80t_lab.gcc2_tempcondset, type:double, comment:null), FieldSchema(name:80t_lab.gcc2_prescond, type:double, comment:null), FieldSchema(name:80t_lab.gcc2_prescondset, type:double, comment:null), FieldSchema(name:80t_lab.pre1_out_p, type:double, comment:null), FieldSchema(name:80t_lab.pre1_inlet_p, type:double, comment:null), FieldSchema(name:80t_lab.pre1_inlet_t, type:double, comment:null), FieldSchema(name:80t_lab.pre1_out_t, type:double, comment:null), FieldSchema(name:80t_lab.gcc4_temprfrg, type:double, comment:null), FieldSchema(name:80t_lab.gcc4_prescond, type:double, comment:null), FieldSchema(name:80t_lab.eev1_reqstate, type:string, comment:null), FieldSchema(name:80t_lab.eev1_actstate, type:string, comment:null), FieldSchema(name:80t_lab.eev1_error_state, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_error_state, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_sensorerror, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_systemalarm, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_systemstategrp, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_press_psig, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_supht_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_sh_setpt_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_valvpos, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_valvposreq, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_valvcntrlen, type:tinyint, comment:null), FieldSchema(name:80t_lab.eev1_vcm_diginput, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_digoutput, type:string, comment:null), FieldSchema(name:80t_lab.eev2_reqstate_byte_1, type:string, comment:null), FieldSchema(name:80t_lab.eev2_actstate_byte_2, type:string, comment:null), FieldSchema(name:80t_lab.eev2_error_state_byte_3, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_error_state_byte_4, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_sensorerror, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_systemalarm, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_systemstategrp, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_press_psig, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_supht_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_sh_setpt_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_valvpos, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_valvposreq, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_valvcntrlen, type:tinyint, comment:null), FieldSchema(name:80t_lab.eev2_vcm_diginput, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_digoutput, type:string, comment:null), FieldSchema(name:80t_lab.pre2_out_p, type:double, comment:null), FieldSchema(name:80t_lab.pre2_inlet_p, type:double, comment:null), FieldSchema(name:80t_lab.pre2_inlet_t, type:double, comment:null), FieldSchema(name:80t_lab.pre2_out_t, type:double, comment:null), FieldSchema(name:80t_lab.eev4_supht, type:double, comment:null), FieldSchema(name:80t_lab.eev4_valvpos, type:tinyint, comment:null), FieldSchema(name:80t_lab.anaoutact1, type:int, comment:null), FieldSchema(name:80t_lab.anaoutact2, type:int, comment:null), FieldSchema(name:80t_lab.anaoutact3, type:int, comment:null), FieldSchema(name:80t_lab.anaoutact4, type:int, comment:null), FieldSchema(name:80t_lab.anain0, type:int, comment:null), FieldSchema(name:80t_lab.anain1, type:int, comment:null), FieldSchema(name:80t_lab.anain2, type:int, comment:null), FieldSchema(name:80t_lab.anain3, type:int, comment:null), FieldSchema(name:80t_lab.anain4, type:int, comment:null), FieldSchema(name:80t_lab.anain5, type:int, comment:null), FieldSchema(name:80t_lab.anain6, type:int, comment:null), FieldSchema(name:80t_lab.anain7, type:int, comment:null), FieldSchema(name:80t_lab.digou1_ss_q01, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou2_ss_q02, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou3_ss_q03, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou9i_ssq04, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou4_ss_q06, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou15_ssq08, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou14_ssq09, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou13_ssq10, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou12_ssq12, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou11_ssq13, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou10_ssq14, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou16_ssq15, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou5_ss_q17, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou6_ss_q18, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou7a_ssk03, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou0w_ssk11, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin0_ss_u15, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin1_ss_u16, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin2_ss, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp0_ssu17, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp1_ssu18, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp2_ssu19, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp3_ssu20, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp4_ssu21, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp5_ssu22, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp6_ssu23, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin13_ssu24, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginau_ssu25, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginhu_ssu26, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp7_ssu27, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin14_ssu28, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin15_ssu29, type:tinyint, comment:null), FieldSchema(name:80t_lab.eev_pre_st_glo, type:string, comment:null), FieldSchema(name:80t_lab.pre1_speedlimitrel, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre2_speedlimitrel, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeoopen, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeost1, type:string, comment:null), FieldSchema(name:80t_lab.peeost2, type:string, comment:null), FieldSchema(name:80t_lab.gcb1_vsdspeedmodemax, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb2_vsdspeedmodemax, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeosof1, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeosof2, type:tinyint, comment:null), FieldSchema(name:80t_lab.fan_glo (q5), type:tinyint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20220131074416_e8fd6276-d028-42d0-9556-6d35bc7e1714); Time taken: 1.01 seconds

avatar
Contributor

@vladenache The issue seems to be with field 

FieldSchema(name:80t_lab.fan_glo (q5), type:tinyint, comment:null)], properties:null)

Please check schema of the table for above field and correct it's name. From the attached stack trace it is trying to identify q5 as a data type and failing as it does not exist 

 

avatar
Explorer

thanks for noticing this, i missed it! 

 

the table is a result of a sqoop job from mysql, so Ii guess the column name had q5 in it and somehow it ended up defined as a type ?!