Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

create table as select failed, but only when stored as parquet

Labels:
avatar
author-rank vladenache
Explorer

Created ‎01-31-2022 05:03 AM

Hello,

 

A simple create table as select fails for some tables in my original_db and I don't know why.

The query I use is below, and the error HIVE gives out is really long, but the main theme is "Vertex Failed", "Error while processing row", "No enum constant" - as you can see below.

 

However, when ran without "STORED AS PARQUET" option, the query runs fine.

Any ideas what I can try to maybe create the tables as compressed parquet?

 

Thanks

 

 

 

beeline -e "SET parquet.compression=SNAPPY;
SET hive.exec.compress.output=true;
SET parquet.output.codec=snappy;
set hive.tez.container.size = 8192;


DROP table if exists DB.table;
CREATE table DB.table
STORED AS parquet

As


SELECT * FROM ORIGINAL_DB.table;"

 

ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1640013186854_27946_1_00, diagnostics=[Task failed, taskId=task_1640013186854_27946_1_00_000004, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1640013186854_27946_1_00_000004_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

[...]

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5

 

5,348 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank tarak271 author-rank
Rising Star

Created ‎02-18-2022 12:59 AM

@vladenache The issue seems to be with field 

FieldSchema(name:80t_lab.fan_glo (q5), type:tinyint, comment:null)], properties:null)

Please check schema of the table for above field and correct it's name. From the attached stack trace it is trying to identify q5 as a data type and failing as it does not exist 

 

View solution in original post

5,244 Views
6 REPLIES 6

avatar
author-rank tarak271 author-rank
Rising Star

Created ‎01-31-2022 10:20 PM

Hi @vladenache 

 

From the pasted stack trace, we could see that enum q5 does not exist. From the code base 

org.apache.parquet.schema.OriginalType

please refer to https://github.com/apache/parquet-mr/blob/master/parquet-column/src/main/java/org/apache/parquet/sch...

So the problem could be with 

  1. Incompatible versions of Parquet/Hive in use, please let us know what versions of hive, parquet are in use or Hadoop distribution version also helps
  2. Source data format issues, please share schema of source table and sample data would help
  3. Please share full stack trace to understand more about code execution path while query is getting executed
5,327 Views

avatar
author-rank vladenache
Explorer

Created on ‎01-31-2022 11:18 PM - edited ‎01-31-2022 11:25 PM

Hello @tarak271 , thanks for your answer 

 

Here are some more details

- Hadoop 3.1.1.7.1.7.63-1

[ I can't manage to get to the parquet version currently in use ]

- the table comes initially from a sqoop job, so maybe some invalid data slipped thru. over 100 columns, so difficult to present them here ( can't attach a sample in my response ) - only 3 types : int, string, double

 

I thought also it might be an invalid value in one of the columns but because I am not returned a row number along with the error message it is impossible to see where that would be.

 

 

 

 

5,317 Views
0 Kudos

avatar
author-rank tarak271 author-rank
Rising Star

Created ‎02-02-2022 12:06 AM

HI @vladenache 

 

Could you please share next 50 lines after the below line

 

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5

 

or full output of beeline console which includes full stack trace

5,296 Views
0 Kudos

avatar
author-rank vladenache
Explorer

Created ‎02-02-2022 12:44 AM

It appears several times in the log

 

Here are the first lines of the error, which include that exception also:

 

ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1640013186854_27946_1_00, diagnostics=[Task failed, taskId=task_1640013186854_27946_1_00_000004, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1640013186854_27946_1_00_000004_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:437)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
	... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
	... 19 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:829)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:1004)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:937)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:937)
	at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
	at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
	... 20 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5
	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:282)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketForFileIdx(FileSinkOperator.java:872)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:823)
	... 27 more
Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.parquet.schema.OriginalType.q5
	at java.lang.Enum.valueOf(Enum.java:238)
	at org.apache.parquet.schema.OriginalType.valueOf(OriginalType.java:24)
	at org.apache.parquet.schema.MessageTypeParser.addPrimitiveType(MessageTypeParser.java:182)
	at org.apache.parquet.schema.MessageTypeParser.addType(MessageTypeParser.java:113)
	at org.apache.parquet.schema.MessageTypeParser.addGroupTypeFields(MessageTypeParser.java:101)
	at org.apache.parquet.schema.MessageTypeParser.parse(MessageTypeParser.java:94)
	at org.apache.parquet.schema.MessageTypeParser.parseMessageType(MessageTypeParser.java:84)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.getSchema(DataWritableWriteSupport.java:51)
	at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.init(DataWritableWriteSupport.java:57)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:418)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:380)
	at org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.<init>(ParquetRecordWriterWrapper.java:70)
	at org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getParquerRecordWriterWrapper(MapredParquetOutputFormat.java:137)
	at org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getHiveRecordWriter(MapredParquetOutputFormat.java:126)
	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getRecordWriter(HiveFileFormatUtils.java:294)
	at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveRecordWriter(HiveFileFormatUtils.java:279)
	... 29 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1640013186854_27946_1_00_000004_1:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:437)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
	... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
	... 19 more

 

Also, here is the schema created

 

INFO  : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:80t_lab.time, type:string, comment:null), FieldSchema(name:80t_lab.trigger, type:int, comment:null), FieldSchema(name:80t_lab.singlstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.temppi_usecfc, type:double, comment:null), FieldSchema(name:80t_lab.humipi_usecfd, type:double, comment:null), FieldSchema(name:80t_lab.locorshtempglo, type:double, comment:null), FieldSchema(name:80t_lab.supply_temp_unfiltered, type:double, comment:null), FieldSchema(name:80t_lab.temp_setpoint, type:double, comment:null), FieldSchema(name:80t_lab.fan_setpoint, type:double, comment:null), FieldSchema(name:80t_lab.temp_local, type:double, comment:null), FieldSchema(name:80t_lab.return_temp, type:double, comment:null), FieldSchema(name:80t_lab.ht_humi, type:double, comment:null), FieldSchema(name:80t_lab.ht_dewpcorr, type:double, comment:null), FieldSchema(name:80t_lab.fc_temp, type:double, comment:null), FieldSchema(name:80t_lab.bara_fan, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_cool, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_deh, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_heate, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_hum, type:tinyint, comment:null), FieldSchema(name:80t_lab.bara_fc, type:tinyint, comment:null), FieldSchema(name:80t_lab.c1a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c1b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c2a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c2b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c3a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c3b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c4a_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.c4b_onoff, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp1, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp2, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp3, type:tinyint, comment:null), FieldSchema(name:80t_lab.manscrramp4, type:tinyint, comment:null), FieldSchema(name:80t_lab.sys_pi_temp, type:double, comment:null), FieldSchema(name:80t_lab.sys_pi_fan, type:double, comment:null), FieldSchema(name:80t_lab.sys_pi_humi, type:double, comment:null), FieldSchema(name:80t_lab.c1_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c2_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c3_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c4_scrolldiff, type:double, comment:null), FieldSchema(name:80t_lab.c1_disct, type:double, comment:null), FieldSchema(name:80t_lab.c2_disct, type:double, comment:null), FieldSchema(name:80t_lab.c3_disct, type:double, comment:null), FieldSchema(name:80t_lab.c4_disct, type:double, comment:null), FieldSchema(name:80t_lab.slp1_status, type:string, comment:null), FieldSchema(name:80t_lab.slp2_status, type:string, comment:null), FieldSchema(name:80t_lab.slp3_status, type:string, comment:null), FieldSchema(name:80t_lab.slp4_status, type:string, comment:null), FieldSchema(name:80t_lab.pb1_status, type:string, comment:null), FieldSchema(name:80t_lab.pb2_status, type:string, comment:null), FieldSchema(name:80t_lab.pb3_status, type:string, comment:null), FieldSchema(name:80t_lab.pb4_status, type:string, comment:null), FieldSchema(name:80t_lab.pb1opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pb2opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pb3opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pb4opspeed, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc1sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc2sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc3sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.gc4sped_f1, type:tinyint, comment:null), FieldSchema(name:80t_lab.leadlag_swap, type:string, comment:null), FieldSchema(name:80t_lab.dxqs_status_glo, type:string, comment:null), FieldSchema(name:80t_lab.pb1_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pb2_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pb3_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pb4_nextst, type:string, comment:null), FieldSchema(name:80t_lab.pre1_pump_request_mode, type:string, comment:null), FieldSchema(name:80t_lab.pre1_pumpstartupstatus, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre1_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre2_pump_request_mode, type:string, comment:null), FieldSchema(name:80t_lab.pre2_pumpstartupstatus, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre2_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre3_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre4_diff_press, type:double, comment:null), FieldSchema(name:80t_lab.pre_early_cmprtomixed, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre_early_mixedtopump, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre_early_cmprtopump, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_tempamb, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_cntrlstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb11_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb12_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb11_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb12_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan1actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan2actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan3actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan1spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan2spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb1_fan3spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcc1_temprfrg, type:double, comment:null), FieldSchema(name:80t_lab.gcc1_tempcondset, type:double, comment:null), FieldSchema(name:80t_lab.gcc1_prescond, type:double, comment:null), FieldSchema(name:80t_lab.gcc1_prescondset, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_temprefrig, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_prescond, type:double, comment:null), FieldSchema(name:80t_lab.gcb1_prescondset, type:double, comment:null), FieldSchema(name:80t_lab.gcb2_tempamb, type:double, comment:null), FieldSchema(name:80t_lab.gcb2_cntrlstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb21_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb22_cntrlssrc, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb21_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb22_compstate, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb2_fan1actual, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb2_fan1spdrqst, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcc2_temprfrg, type:double, comment:null), FieldSchema(name:80t_lab.gcc2_tempcondset, type:double, comment:null), FieldSchema(name:80t_lab.gcc2_prescond, type:double, comment:null), FieldSchema(name:80t_lab.gcc2_prescondset, type:double, comment:null), FieldSchema(name:80t_lab.pre1_out_p, type:double, comment:null), FieldSchema(name:80t_lab.pre1_inlet_p, type:double, comment:null), FieldSchema(name:80t_lab.pre1_inlet_t, type:double, comment:null), FieldSchema(name:80t_lab.pre1_out_t, type:double, comment:null), FieldSchema(name:80t_lab.gcc4_temprfrg, type:double, comment:null), FieldSchema(name:80t_lab.gcc4_prescond, type:double, comment:null), FieldSchema(name:80t_lab.eev1_reqstate, type:string, comment:null), FieldSchema(name:80t_lab.eev1_actstate, type:string, comment:null), FieldSchema(name:80t_lab.eev1_error_state, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_error_state, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_sensorerror, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_systemalarm, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_systemstategrp, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_press_psig, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_supht_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_sh_setpt_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_valvpos, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_valvposreq, type:double, comment:null), FieldSchema(name:80t_lab.eev1_vcm_valvcntrlen, type:tinyint, comment:null), FieldSchema(name:80t_lab.eev1_vcm_diginput, type:string, comment:null), FieldSchema(name:80t_lab.eev1_vcm_digoutput, type:string, comment:null), FieldSchema(name:80t_lab.eev2_reqstate_byte_1, type:string, comment:null), FieldSchema(name:80t_lab.eev2_actstate_byte_2, type:string, comment:null), FieldSchema(name:80t_lab.eev2_error_state_byte_3, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_error_state_byte_4, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_sensorerror, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_systemalarm, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_systemstategrp, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_press_psig, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_supht_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_sh_setpt_degf, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_valvpos, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_valvposreq, type:double, comment:null), FieldSchema(name:80t_lab.eev2_vcm_valvcntrlen, type:tinyint, comment:null), FieldSchema(name:80t_lab.eev2_vcm_diginput, type:string, comment:null), FieldSchema(name:80t_lab.eev2_vcm_digoutput, type:string, comment:null), FieldSchema(name:80t_lab.pre2_out_p, type:double, comment:null), FieldSchema(name:80t_lab.pre2_inlet_p, type:double, comment:null), FieldSchema(name:80t_lab.pre2_inlet_t, type:double, comment:null), FieldSchema(name:80t_lab.pre2_out_t, type:double, comment:null), FieldSchema(name:80t_lab.eev4_supht, type:double, comment:null), FieldSchema(name:80t_lab.eev4_valvpos, type:tinyint, comment:null), FieldSchema(name:80t_lab.anaoutact1, type:int, comment:null), FieldSchema(name:80t_lab.anaoutact2, type:int, comment:null), FieldSchema(name:80t_lab.anaoutact3, type:int, comment:null), FieldSchema(name:80t_lab.anaoutact4, type:int, comment:null), FieldSchema(name:80t_lab.anain0, type:int, comment:null), FieldSchema(name:80t_lab.anain1, type:int, comment:null), FieldSchema(name:80t_lab.anain2, type:int, comment:null), FieldSchema(name:80t_lab.anain3, type:int, comment:null), FieldSchema(name:80t_lab.anain4, type:int, comment:null), FieldSchema(name:80t_lab.anain5, type:int, comment:null), FieldSchema(name:80t_lab.anain6, type:int, comment:null), FieldSchema(name:80t_lab.anain7, type:int, comment:null), FieldSchema(name:80t_lab.digou1_ss_q01, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou2_ss_q02, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou3_ss_q03, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou9i_ssq04, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou4_ss_q06, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou15_ssq08, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou14_ssq09, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou13_ssq10, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou12_ssq12, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou11_ssq13, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou10_ssq14, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou16_ssq15, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou5_ss_q17, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou6_ss_q18, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou7a_ssk03, type:tinyint, comment:null), FieldSchema(name:80t_lab.digou0w_ssk11, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin0_ss_u15, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin1_ss_u16, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin2_ss, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp0_ssu17, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp1_ssu18, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp2_ssu19, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp3_ssu20, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp4_ssu21, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp5_ssu22, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp6_ssu23, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin13_ssu24, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginau_ssu25, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginhu_ssu26, type:tinyint, comment:null), FieldSchema(name:80t_lab.diginp7_ssu27, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin14_ssu28, type:tinyint, comment:null), FieldSchema(name:80t_lab.digin15_ssu29, type:tinyint, comment:null), FieldSchema(name:80t_lab.eev_pre_st_glo, type:string, comment:null), FieldSchema(name:80t_lab.pre1_speedlimitrel, type:tinyint, comment:null), FieldSchema(name:80t_lab.pre2_speedlimitrel, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeoopen, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeost1, type:string, comment:null), FieldSchema(name:80t_lab.peeost2, type:string, comment:null), FieldSchema(name:80t_lab.gcb1_vsdspeedmodemax, type:tinyint, comment:null), FieldSchema(name:80t_lab.gcb2_vsdspeedmodemax, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeosof1, type:tinyint, comment:null), FieldSchema(name:80t_lab.peeosof2, type:tinyint, comment:null), FieldSchema(name:80t_lab.fan_glo (q5), type:tinyint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20220131074416_e8fd6276-d028-42d0-9556-6d35bc7e1714); Time taken: 1.01 seconds
5,290 Views
0 Kudos

avatar
author-rank tarak271 author-rank
Rising Star

Created ‎02-18-2022 12:59 AM

@vladenache The issue seems to be with field 

FieldSchema(name:80t_lab.fan_glo (q5), type:tinyint, comment:null)], properties:null)

Please check schema of the table for above field and correct it's name. From the attached stack trace it is trying to identify q5 as a data type and failing as it does not exist 

 

5,245 Views

avatar
author-rank vladenache
Explorer

Created ‎02-18-2022 03:45 AM

thanks for noticing this, i missed it! 

 

the table is a result of a sqoop job from mysql, so Ii guess the column name had q5 in it and somehow it ended up defined as a type ?!

 

 

5,233 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements