Support Questions

Find answers, ask questions, and share your expertise

CDP - Facing issues with Hive

avatar
Contributor

Hello Team,

 

Facing bellow issues with hive while loading data from HDFS.

Following are the error:

 

Error 1

ERROR : Vertex failed, vertexName=Reducer 2, vertexId=vertex_1672408587164_0154_2_01, diagnostics=[Task failed, taskId=task_1672408587164_0154_2_01_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1672408587164_0154_2_01_000001_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing vector batch (tag=0) (vectorizedVertexNum 1)



Error 2

Caused by: org.apache.hadoop.hive.ql.metadata.HiveFatalException: [Error 20004]: Fatal error occurred when node tried to create too many dynamic partitions. The maximum number of dynamic partitions is controlled by hive.exec.max.dynamic.partitions and hive.exec.max.dynamic.partitions.pernode. Maximum was set to 2000 partitions per node, number of dynamic partitions on this node: 2001

 

Error 3

ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1672408587164_0175_1_00, diagnostics=[Vertex vertex_1672408587164_0175_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: offer_15__temp_table_for_load_data__ initializer failed, vertex=vertex_1672408587164_0175_1_00 [Map 1], java.lang.RuntimeException: ORC split generation failed with exception: java.io.IOException: Illegal type id 0. The valid range is 0 to -1

 

Please suggest the resolution for above errors.Thanks.

1 REPLY 1

avatar
Master Collaborator

The stack traces for Error 1 and Error 3 are incomplete. To gain a better understanding of the issue, please provide the complete stack traces. Sharing the complete appLogs will provide a comprehensive view of the situation

Regarding error 2, it appears that the job is attempting to create over 2000 dynamic partitions on a single node, which is an unusual behavior. Please review the partition column values for correctness. If everything appears to be in order, you can consider adjusting the following configurations:

  • hive.exec.max.dynamic.partitions
  • hive.exec.max.dynamic.partitions.pernode