Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

[#1]

Please find the below screenshot and attached logs.

Error:

SEVERE: The Integration Service failed to run the Hive task [Write_test_1jiq2sdd]. See the additional error messages for more information.

com.informatica.sdk.dtm.ExecutionException: [[HIVE_1070] The Integration Service failed to run Hive query [Write_test_1jiq2sdd_query_3] for task [Write_test_1jiq2sdd] due to following error: Hive error code [2], Hive message [Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1764063775281_0054_1_00, diagnostics=[Vertex vertex_1764063775281_0054_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, java.util.concurrent.ExecutionException: org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator

2025-11-26 14:25:24,278 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: HADOOP_MAPRED_HOME is /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib

2025-11-26 14:25:24,524 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: Could not rename /tmp/sqoop-infadpdev/887867565228272/vrp_branch.java to /tmp/sqoop-infadpdev/887867565228272/vrp_branch.java. Error: File element in parameter 'null' already exists: '/tmp/sqoop-infadpdev/887867565228272/vrp_branch.java'

2025-11-26 14:25:24,524 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: Writing jar file: /tmp/sqoop-infadpdev/887867565228272/vrp_branch.jar

2025-11-26 14:25:24,655 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: Destination directory hdfs://saibdev/data_lakehouse/tables/raw/SPARK_k164prda/sqoop_staging/S7943081602786963440/CUSOMTER_DO_SRC_20698ca1de97472bbab0dfa0331a4a93 is not present, hence not deleting.

_write_ing_m_dynamic_create.vrp_BranchSubnet as a58, w7943081602786963440_write_ing_m_dynamic_create.vrp_BranchTypeCode as a59, w7943081602786963440_write_ing_m_dynamic_create.vrp_DefaultOfficerCode as a60, w7943081602786963440_write_ing_m_dynamic_create.vrp_DefaultSundryCode as a61, w7943081602786963440_write_ing_m_dynamic_create.vrp_Reserved as a62, w7943081602786963440_write_ing_m_dynamic_create.vrp_Unitmnemonic as a63, w7943081602786963440_write_ing_m_dynamic_create.vrp_Zipcode as a64, w7943081602786963440_write_ing_m_dynamic_create.vrp_SalesQueue as a65, w7943081602786963440_write_ing_m_dynamic_create.vrp_BranchApproverQueue as a66, w7943081602786963440_write_ing_m_dynamic_create.vrp_CustomerClassification as a67, w7943081602786963440_write_ing_m_dynamic_create.vrp_SeniorBSRQueue as a68, w7943081602786963440_write_ing_m_dynamic_create.vrp_seniorbsrteam as a69, w7943081602786963440_write_ing_m_dynamic_create.vrp_BranchAccountNumber as a70 FROM default.w7943081602786963440_write_ing_m_dynamic_create

2025-11-26 14:31:10,660 AST <LdtmWorkflowTask-pool-6-thread-4> WARNING: java.sql.SQLException: Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1764063775281_0133_1_00, diagnostics=[Vertex vertex_1764063775281_0133_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, java.util.concurrent.ExecutionException: org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator

at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:592)

at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:571)

at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:91)

at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializerWrappers(RootInputInitializerManager.java:140)

at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:111)

at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4147)

at org.apache.tez.dag.app.dag.impl.VertexImpl.access$3100(VertexImpl.java:210)

Caused by: java.lang.ExceptionInInitializerError

at org.apache.hadoop.hive.ql.plan.TableDesc.setProperties(TableDesc.java:131)

at org.apache.hadoop.hive.ql.plan.TableDesc.<init>(TableDesc.java:69)

at org.apache.hadoop.hive.ql.exec.Utilities.<clinit>(Utilities.java:706)

at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:150)

... 21 more

[#2]

@Amr5 
From the logs you shared the core issue is 

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask
Vertex failed: INIT_FAILURE
Unable to instantiate class with 1 arguments:org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
Caused by: java.lang.ExceptionInInitializerError

This indicates a classpath/library compatibility issue between Informatica and the upgraded Cloudera CDP cluster, specifically with Tez and Hive components. 
Root Causes:

1. Version Mismatch: The Informatica integration is pointing to CDH 7.218 libraries, your HADOOP_MAPRED_HOME is /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib  but your cluster was upgraded to a newer CDP version with incompatible Hive/Tez
   libraries.
2. Class Initialization Failure: The HiveSplitGenerator class cannot be instantiated, likely due to missing or incompatible dependencies.
3. Static Initializer Problem: The  ExceptionInInitializerError suggests a static block in one of the Hive classes is failing during initialization.

Solution 1
Update the Informatica Hadoop connection:

• Go to Administrator → Connections
• Edit your Hadoop connection
• Update the Hadoop distribution version to match your new CDP version
• Update the configuration files (core-site.xml, hdfs-site.xml, hive-site.xml, etc.)

Update Hadoop libraries:
Copy new CDP client libraries to Informatica

Spoiler

cp -r /opt/cloudera/parcels/CDH/lib/* /data1/informatica/dei/services/shared/hadoop/CDP_<version>/lib/

Restart Informatica services:

Spoiler

infaservice.sh stopService
infaservice.sh startService

Additionally Identify and copy missing Tez JARs

Spoiler

# From CDP cluster, copy Tez libraries
cp /opt/cloudera/parcels/CDH/lib/tez/*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/
# Copy Hive execution libraries
cp /opt/cloudera/parcels/CDH/lib/hive/lib/hive-exec-*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/

Update classpathin Informatica domain configuration
Configure Hive Execution Engine If Tez is causing issues, temporarily switch to MapReduce.

In your Hive connection properties, add:

Spoiler

hive.execution.engine=mr

Update this post after the above steps and always share the logs to enable us understand whats happening in your environment.

Happy hadooping

[#3]

@Shelton ,

First Many thanks for sharing your insights with me much appreciated.

we have re run this workflow again from informatica now again with some new details same errors are coming.

Logs:

SEVERE: The Integration Service failed to run the Hive task [Write_test_76ti7ctz]. See the additional error messages for more information.
com.informatica.sdk.dtm.ExecutionException: [[HIVE_1070] The Integration Service failed to run Hive query [Write_test_76ti7ctz_query_3] for task [Write_test_76ti7ctz] due to following error: Hive error code [2], Hive message [Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1764836049228_0036_1_00, diagnostics=[Vertex vertex_1764836049228_0036_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, java.util.concurrent.ExecutionException: org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator

2025-12-09 16:44:25,009 AST <LdtmWorkflowTask-pool-6-thread-8> INFO: Could not rename /tmp/sqoop-infadpdev/673817950266090/Fact_Bills_DD.java to /tmp/sqoop-infadpdev/673817950266090/Fact_Bills_DD.java. Error: File element in parameter 'null' already exists: '/tmp/sqoop-infadpdev/673817950266090/Fact_Bills_DD.java'

2025-12-09 16:44:21,704 AST <MappingCompiler-pool-4-thread-3> INFO: [LDTMCMN_0037] The Hadoop distribution directory is defined in Data Integration Service properties at the path [/data1/informatica/dei/services/shared/hadoop/CDH_7.218].
2025-12-09 16:44:21,704 AST <MappingCompiler-pool-4-thread-3> INFO: [CLUSTERCONF_10024] The cluster configuration [Cloudera_Dev] is unchanged from the last export. Using the existing export file [/data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/SPARK/665d244e-0368-4d51-8ac9-01e1ac851a1f/infacco-site.xml].
2025-12-09 16:44:21,704 AST <MappingCompiler-pool-4-thread-3> INFO: [CLUSTERCONF_10028] Based on the distribution [CLOUDERA] and the run-time engine [SPARK], the Data Integration Service will override the following cluster configuration properties at run time: \n  - fs.file.impl.disable.cache: true\n  - fs.hdfs.impl.disable.cache: true\n  - yarn.timeline-service.enabled: false

--------------------

run this and locate command for hive-exec.jar on informatica server and see that all hive-exec.jars were on older cloudera version which was 7.1.9

sudo find / -type f -name "hive-exec.jar" 2>/dev/null

 Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator is in hive-exec.jar

which is pointing to old path.

[#4]

@Amr5 
Just as you too realized  there is an old path issue. The error indicates a version mismatch between the Hive/Tez libraries being used by Informatica and those expected by your Cloudera cluster.
RCA
1. Informatica is using Hive libraries from an older Cloudera version (7.1.9)
2. Your cluster is running Cloudera 7.2.18 (as shown in the path   /data1/informatica/dei/services/shared/hadoop/CDH_7.218
3. The HiveSplitGenerator class in the old hive-exec.jar is incompatible with the newer Tez runtime
Step 1: Locate Current Hive Libraries

Spoiler

sudo find /data1/informatica -type f -name "hive-exec*.jar" 2>/dev/null

Step 2: Backup Old Libraries

Spoiler

cd /data1/informatica/dei/services/shared/hadoop/CDH_7.218
mkdir -p backup_old_hive_libs
mv hive-exec*.jar backup_old_hive_libs/

Step 3: Copy Correct Hive Libraries from Cluster

Spoiler

# Find the correct hive-exec.jar on your Cloudera cluster
find /opt/cloudera/parcels -name "hive-exec*.jar" 2>/dev/null

# Copy it to Informatica's Hadoop distribution directory
cp /opt/cloudera/parcels/CDH-7.2.18*/lib/hive/lib/hive-exec-*.jar \
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/

Step 4: Update Informatica Hadoop Distribution

In Informatica Administrator Console:

1. Navigate to Data Integration Service → Properties
2. Go to Hadoop Connection → Distribution
3. Verify it points to: /data1/informatica/dei/services/shared/hadoop/CDH_7.218
4. Click Test Connection to validate
5. If needed, use Re-import Hadoop Configuration to refresh cluster configs

Step 5: Restart Services

Spoiler

infaservice.sh dis restart -domain DOM_IDQ_DEV -service DIS_DEI_DEV

Step 6: Clear Cached Compilation Files

Spoiler

rm -rf /data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/SPARK/*
rm -rf /tmp/sqoop-infadpdev/*

Step 7: Re-run Your Mapping
If you have multiple nodes in your Informatica cluster, repeat Steps 2-3 on all nodes where the Data Integration Service runs.

Happy hadooping

[#5]

@Shelton 

after following shared steps mentioned, we re run the mapping getting this error now:

java.lang.NoSuchMethodError:
'org.apache.hadoop.hive.ql.parse.ParseResult org.apache.hadoop.hive.ql.parse.ParseDriver.parse(java.lang.String, org.apache.hadoop.conf.Configuration)'

please also note this that old jars for hive-exec also there in path :
/data1/informatica/dei/services/shared/hadoop/CDH_7.218

data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.3.1.100-57.jar
/data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.2.18.0-641.jar
*****/data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.1/lib/hive-exec-3.1.3000.7.1.9.0-387.jar

/data1/informatica/dei/externaljdbcjars/hive-exec-3.1.3000.7.3.1.100-57.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.3.1.100-57.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.3.1.100-57.jar
*** /data1/informatica/dei/services/shared/hadoop/CDH_7.1/lib/hive-exec-3.1.3000.7.1.9.0-387.jar
*** /data1/informatica/dei/services/shared/hadoop/CDH_7.1/spark/jars/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/spark/lib_spark/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/spark/lib_spark/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/services/shared/spark/lib_spark/hive-exec-3.1.3000.7.3.1.100-57.jar

[#6]

@Amr5 

The NoSuchMethodError means JAR conflict at runtime. You must ensure that only CDH 7.2.18 Hive JARs are in the classpath, with no remnants of 7.1.9.
The ParseDriver.parse() method signature changed between Hive versions. In your case the Old Hive JARs (from CDH 7.1.9) are still present in /data1/informatica/dei/services/shared/hadoop/CDH_7.218
Java is loading the old hive-exec.jar instead of the new one, causing method signature mismatches
Step 1. Identify ALL Old Hive JARs

Spoiler

find /data1/informatica/dei/services/shared/hadoop/CDH_7.218 -name "hive*.jar" -exec ls -lh {} \;

Step 2: Remove ALL Old Hive JARs

Spoiler

cd /data1/informatica/dei/services/shared/hadoop/CDH_7.218
# Create backup directory if not exists
mkdir -p backup_all_old_hive_jars
# Move ALL hive-related JARs to backup
mv hive*.jar backup_all_old_hive_jars/

Step 3: Copy ALL Correct Hive JARs from Cloudera Cluster

Spoiler

# Find Cloudera CDH 7.2.18 parcels location
CLOUDERA_PARCEL=$(find /opt/cloudera/parcels -maxdepth 1 -type d -name "CDH-7.2.18*" | head -1)
# Copy ALL Hive JARs
cp $CLOUDERA_PARCEL/lib/hive/lib/hive*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/
# Also copy Hive dependencies
cp $CLOUDERA_PARCEL/jars/hive*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/

Step 4: Verify Correct Versions

Spoiler

cd /data1/informatica/dei/services/shared/hadoop/CDH_7.218
ls -lh hive*.jar | head -5

# Check the version inside hive-exec.jar
unzip -p hive-exec-*.jar META-INF/MANIFEST.MF | grep -i version

Step 5: Clear Java Classpath Cache

Spoiler

# Remove compiled artifacts
rm -rf /data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/SPARK/*
rm -rf /data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/HIVE/*

Step 6: Restart Informatica Services

Spoiler

infaservice.sh dis stop -domain DOM_IDQ_DEV -service DIS_DEI_DEV
infaservice.sh dis start -domain DOM_IDQ_DEV -service DIS_DEI_DEV

Step 7: Verify Hadoop Distribution in Informatica Admin Console

1. Login to Informatica Administrator
2. Navigate to DIS_DEI_DEV → Properties → Hadoop Connection
3. Click Test Connection
4. If it fails, click Re-import Hadoop Configuration to refresh

Step 8: Re-run Your Mapping

Happy Hadooping