question Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster in Support Questions

Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Amr5 — Tue, 21 Apr 2026 06:11:42 GMT

Please find the below screenshot and attached logs.

Error:

SEVERE: The Integration Service failed to run the Hive task [Write_test_1jiq2sdd]. See the additional error messages for more information.

com.informatica.sdk.dtm.ExecutionException: [[HIVE_1070] The Integration Service failed to run Hive query [Write_test_1jiq2sdd_query_3] for task [Write_test_1jiq2sdd] due to following error: Hive error code [2], Hive message [Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1764063775281_0054_1_00, diagnostics=[Vertex vertex_1764063775281_0054_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, java.util.concurrent.ExecutionException: org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator

2025-11-26 14:25:24,278 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: HADOOP_MAPRED_HOME is /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib

2025-11-26 14:25:24,524 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: Could not rename /tmp/sqoop-infadpdev/887867565228272/vrp_branch.java to /tmp/sqoop-infadpdev/887867565228272/vrp_branch.java. Error: File element in parameter 'null' already exists: '/tmp/sqoop-infadpdev/887867565228272/vrp_branch.java'

2025-11-26 14:25:24,524 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: Writing jar file: /tmp/sqoop-infadpdev/887867565228272/vrp_branch.jar

2025-11-26 14:25:24,655 AST <LdtmWorkflowTask-pool-6-thread-3> INFO: Destination directory hdfs://saibdev/data_lakehouse/tables/raw/SPARK_k164prda/sqoop_staging/S7943081602786963440/CUSOMTER_DO_SRC_20698ca1de97472bbab0dfa0331a4a93 is not present, hence not deleting.

_write_ing_m_dynamic_create.vrp_BranchSubnet as a58, w7943081602786963440_write_ing_m_dynamic_create.vrp_BranchTypeCode as a59, w7943081602786963440_write_ing_m_dynamic_create.vrp_DefaultOfficerCode as a60, w7943081602786963440_write_ing_m_dynamic_create.vrp_DefaultSundryCode as a61, w7943081602786963440_write_ing_m_dynamic_create.vrp_Reserved as a62, w7943081602786963440_write_ing_m_dynamic_create.vrp_Unitmnemonic as a63, w7943081602786963440_write_ing_m_dynamic_create.vrp_Zipcode as a64, w7943081602786963440_write_ing_m_dynamic_create.vrp_SalesQueue as a65, w7943081602786963440_write_ing_m_dynamic_create.vrp_BranchApproverQueue as a66, w7943081602786963440_write_ing_m_dynamic_create.vrp_CustomerClassification as a67, w7943081602786963440_write_ing_m_dynamic_create.vrp_SeniorBSRQueue as a68, w7943081602786963440_write_ing_m_dynamic_create.vrp_seniorbsrteam as a69, w7943081602786963440_write_ing_m_dynamic_create.vrp_BranchAccountNumber as a70 FROM default.w7943081602786963440_write_ing_m_dynamic_create

2025-11-26 14:31:10,660 AST <LdtmWorkflowTask-pool-6-thread-4> WARNING: java.sql.SQLException: Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1764063775281_0133_1_00, diagnostics=[Vertex vertex_1764063775281_0133_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, java.util.concurrent.ExecutionException: org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator

at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:592)

at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:571)

at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:91)

at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializerWrappers(RootInputInitializerManager.java:140)

at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:111)

at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4147)

at org.apache.tez.dag.app.dag.impl.VertexImpl.access$3100(VertexImpl.java:210)

Caused by: java.lang.ExceptionInInitializerError

at org.apache.hadoop.hive.ql.plan.TableDesc.setProperties(TableDesc.java:131)

at org.apache.hadoop.hive.ql.plan.TableDesc.<init>(TableDesc.java:69)

at org.apache.hadoop.hive.ql.exec.Utilities.<clinit>(Utilities.java:706)

at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:150)

... 21 more

Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Shelton — Mon, 08 Dec 2025 07:30:08 GMT

@Amr5
From the logs you shared the core issue is

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask Vertex failed: INIT_FAILURE Unable to instantiate class with 1 arguments:org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator Caused by: java.lang.ExceptionInInitializerError

This indicates a classpath/library compatibility issue between Informatica and the upgraded Cloudera CDP cluster, specifically with Tez and Hive components.
Root Causes:

Version Mismatch: The Informatica integration is pointing to CDH 7.218 libraries, your HADOOP_MAPRED_HOME is /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib but your cluster was upgraded to a newer CDP version with incompatible Hive/Tez
libraries.
Class Initialization Failure: The HiveSplitGenerator class cannot be instantiated, likely due to missing or incompatible dependencies.
Static Initializer Problem: The ExceptionInInitializerError suggests a static block in one of the Hive classes is failing during initialization.

Solution 1
Update the Informatica Hadoop connection:

Go to Administrator → Connections
Edit your Hadoop connection
Update the Hadoop distribution version to match your new CDP version
Update the configuration files (core-site.xml, hdfs-site.xml, hive-site.xml, etc.)

Update Hadoop libraries:
Copy new CDP client libraries to Informatica

cp -r /opt/cloudera/parcels/CDH/lib/* /data1/informatica/dei/services/shared/hadoop/CDP_<version>/lib/

Restart Informatica services:

infaservice.sh stopService
infaservice.sh startService

Additionally Identify and copy missing Tez JARs

# From CDP cluster, copy Tez libraries
cp /opt/cloudera/parcels/CDH/lib/tez/*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/
# Copy Hive execution libraries
cp /opt/cloudera/parcels/CDH/lib/hive/lib/hive-exec-*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/

Update classpathin Informatica domain configuration
Configure Hive Execution Engine If Tez is causing issues, temporarily switch to MapReduce.

In your Hive connection properties, add:

hive.execution.engine=mr

Update this post after the above steps and always share the logs to enable us understand whats happening in your environment.

Happy hadooping

Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Amr5 — Tue, 09 Dec 2025 15:10:41 GMT

@Shelton ,

First Many thanks for sharing your insights with me much appreciated.

we have re run this workflow again from informatica now again with some new details same errors are coming.

Logs:

SEVERE: The Integration Service failed to run the Hive task [Write_test_76ti7ctz]. See the additional error messages for more information.
com.informatica.sdk.dtm.ExecutionException: [[HIVE_1070] The Integration Service failed to run Hive query [Write_test_76ti7ctz_query_3] for task [Write_test_76ti7ctz] due to following error: Hive error code [2], Hive message [Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1764836049228_0036_1_00, diagnostics=[Vertex vertex_1764836049228_0036_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, java.util.concurrent.ExecutionException: org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator

2025-12-09 16:44:25,009 AST <LdtmWorkflowTask-pool-6-thread-8> INFO: Could not rename /tmp/sqoop-infadpdev/673817950266090/Fact_Bills_DD.java to /tmp/sqoop-infadpdev/673817950266090/Fact_Bills_DD.java. Error: File element in parameter 'null' already exists: '/tmp/sqoop-infadpdev/673817950266090/Fact_Bills_DD.java'

2025-12-09 16:44:21,704 AST <MappingCompiler-pool-4-thread-3> INFO: [LDTMCMN_0037] The Hadoop distribution directory is defined in Data Integration Service properties at the path [/data1/informatica/dei/services/shared/hadoop/CDH_7.218].
2025-12-09 16:44:21,704 AST <MappingCompiler-pool-4-thread-3> INFO: [CLUSTERCONF_10024] The cluster configuration [Cloudera_Dev] is unchanged from the last export. Using the existing export file [/data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/SPARK/665d244e-0368-4d51-8ac9-01e1ac851a1f/infacco-site.xml].
2025-12-09 16:44:21,704 AST <MappingCompiler-pool-4-thread-3> INFO: [CLUSTERCONF_10028] Based on the distribution [CLOUDERA] and the run-time engine [SPARK], the Data Integration Service will override the following cluster configuration properties at run time: \n - fs.file.impl.disable.cache: true\n - fs.hdfs.impl.disable.cache: true\n - yarn.timeline-service.enabled: false

--------------------

run this and locate command for hive-exec.jar on informatica server and see that all hive-exec.jars were on older cloudera version which was 7.1.9

sudo find / -type f -name "hive-exec.jar" 2>/dev/null

Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator is in hive-exec.jar

which is pointing to old path.

Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Shelton — Tue, 09 Dec 2025 15:56:25 GMT

@Amr5
Just as you too realized there is an old path issue. The error indicates a version mismatch between the Hive/Tez libraries being used by Informatica and those expected by your Cloudera cluster.
RCA
1. Informatica is using Hive libraries from an older Cloudera version (7.1.9)
2. Your cluster is running Cloudera 7.2.18 (as shown in the path /data1/informatica/dei/services/shared/hadoop/CDH_7.218
3. The HiveSplitGenerator class in the old hive-exec.jar is incompatible with the newer Tez runtime
Step 1: Locate Current Hive Libraries

sudo find /data1/informatica -type f -name "hive-exec*.jar" 2>/dev/null

Step 2: Backup Old Libraries

cd /data1/informatica/dei/services/shared/hadoop/CDH_7.218
mkdir -p backup_old_hive_libs
mv hive-exec*.jar backup_old_hive_libs/

Step 3: Copy Correct Hive Libraries from Cluster

# Find the correct hive-exec.jar on your Cloudera cluster
find /opt/cloudera/parcels -name "hive-exec*.jar" 2>/dev/null

# Copy it to Informatica's Hadoop distribution directory
cp /opt/cloudera/parcels/CDH-7.2.18*/lib/hive/lib/hive-exec-*.jar \
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/

Step 4: Update Informatica Hadoop Distribution

In Informatica Administrator Console:

Navigate to Data Integration Service → Properties
Go to Hadoop Connection → Distribution
Verify it points to: /data1/informatica/dei/services/shared/hadoop/CDH_7.218
Click Test Connection to validate
If needed, use Re-import Hadoop Configuration to refresh cluster configs

Step 5: Restart Services

infaservice.sh dis restart -domain DOM_IDQ_DEV -service DIS_DEI_DEV

Step 6: Clear Cached Compilation Files

rm -rf /data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/SPARK/*
rm -rf /tmp/sqoop-infadpdev/*

Step 7: Re-run Your Mapping
If you have multiple nodes in your Informatica cluster, repeat Steps 2-3 on all nodes where the Data Integration Service runs.

Happy hadooping

Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Amr5 — Wed, 10 Dec 2025 12:04:03 GMT

@Shelton

after following shared steps mentioned, we re run the mapping getting this error now:

java.lang.NoSuchMethodError:
'org.apache.hadoop.hive.ql.parse.ParseResult org.apache.hadoop.hive.ql.parse.ParseDriver.parse(java.lang.String, org.apache.hadoop.conf.Configuration)'

please also note this that old jars for hive-exec also there in path :
/data1/informatica/dei/services/shared/hadoop/CDH_7.218

data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.3.1.100-57.jar
/data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.2.18.0-641.jar
*****/data1/informatica/dei/clients/DeveloperClient/hadoop/CDH_7.1/lib/hive-exec-3.1.3000.7.1.9.0-387.jar

/data1/informatica/dei/externaljdbcjars/hive-exec-3.1.3000.7.3.1.100-57.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/lib/hive-exec-3.1.3000.7.3.1.100-57.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/services/shared/hadoop/CDH_7.218/spark/jars/hive-exec-3.1.3000.7.3.1.100-57.jar
*** /data1/informatica/dei/services/shared/hadoop/CDH_7.1/lib/hive-exec-3.1.3000.7.1.9.0-387.jar
*** /data1/informatica/dei/services/shared/hadoop/CDH_7.1/spark/jars/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/spark/lib_spark/hive-exec-3.1.3000.7.1.9.0-387.jar
/data1/informatica/dei/services/shared/spark/lib_spark/hive-exec-3.1.3000.7.2.18.0-641.jar
/data1/informatica/dei/services/shared/spark/lib_spark/hive-exec-3.1.3000.7.3.1.100-57.jar

Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Shelton — Wed, 10 Dec 2025 12:21:52 GMT

@Amr5

The NoSuchMethodError means JAR conflict at runtime. You must ensure that only CDH 7.2.18 Hive JARs are in the classpath, with no remnants of 7.1.9.
The ParseDriver.parse() method signature changed between Hive versions. In your case the Old Hive JARs (from CDH 7.1.9) are still present in /data1/informatica/dei/services/shared/hadoop/CDH_7.218
Java is loading the old hive-exec.jar instead of the new one, causing method signature mismatches
Step 1. Identify ALL Old Hive JARs

find /data1/informatica/dei/services/shared/hadoop/CDH_7.218 -name "hive*.jar" -exec ls -lh {} \;

Step 2: Remove ALL Old Hive JARs

cd /data1/informatica/dei/services/shared/hadoop/CDH_7.218
# Create backup directory if not exists
mkdir -p backup_all_old_hive_jars
# Move ALL hive-related JARs to backup
mv hive*.jar backup_all_old_hive_jars/

Step 3: Copy ALL Correct Hive JARs from Cloudera Cluster

# Find Cloudera CDH 7.2.18 parcels location
CLOUDERA_PARCEL=$(find /opt/cloudera/parcels -maxdepth 1 -type d -name "CDH-7.2.18*" | head -1)
# Copy ALL Hive JARs
cp $CLOUDERA_PARCEL/lib/hive/lib/hive*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/
# Also copy Hive dependencies
cp $CLOUDERA_PARCEL/jars/hive*.jar /data1/informatica/dei/services/shared/hadoop/CDH_7.218/

Step 4: Verify Correct Versions

cd /data1/informatica/dei/services/shared/hadoop/CDH_7.218
ls -lh hive*.jar | head -5

# Check the version inside hive-exec.jar
unzip -p hive-exec-*.jar META-INF/MANIFEST.MF | grep -i version

Step 5: Clear Java Classpath Cache

# Remove compiled artifacts
rm -rf /data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/SPARK/*
rm -rf /data1/informatica/dei/tomcat/bin/disTemp/DOM_IDQ_DEV/DIS_DEI_DEV/node02_DEI_DEV/cloudera_dev/HIVE/*

Step 6: Restart Informatica Services

infaservice.sh dis stop -domain DOM_IDQ_DEV -service DIS_DEI_DEV
infaservice.sh dis start -domain DOM_IDQ_DEV -service DIS_DEI_DEV

Step 7: Verify Hadoop Distribution in Informatica Admin Console

Login to Informatica Administrator
Navigate to DIS_DEI_DEV → Properties → Hadoop Connection
Click Test Connection
If it fails, click Re-import Hadoop Configuration to refresh

Step 8: Re-run Your Mapping

Happy Hadooping

Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Amr5 — Sun, 21 Dec 2025 10:15:43 GMT

Hi again,

We tried re run after updating jars, raised with infa support also, they suggested to roll back old path which CDH 7.2.18 in infa ,

and follow this KB : https://knowledge.informatica.com/s/article/000251302?language=en_US

add these :
In order to resolve this error add below post sql query in Hive target:

set tez.am.launch.cmd-opts="-XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -Xmx20480m \

--add-opens=java.base/java.net=ALL-UNNAMED \

--add-opens=java.base/java.util=ALL-UNNAMED \

--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED \

--add-opens=java.base/java.util.regex=ALL-UNNAMED \

--add-opens=java.base/java.lang=ALL-UNNAMED \

--add-opens=java.base/java.time=ALL-UNNAMED \

--add-opens=java.base/java.io=ALL-UNNAMED \

--add-opens=java.base/java.nio=ALL-UNNAMED";

Re: Hive execution stage failing in informatica - after upgrading cloudera cdp cluster

Amr5 — Thu, 25 Dec 2025 08:35:10 GMT

@Shelton still same
JDK version used in the Hadoop cluster( for both 7.1.9 & 7.3.1 ).

dev - 7.3.1 -> java --version

openjdk 17.0.16 2025-07-15 LTS
OpenJDK Runtime Environment (Red_Hat-17.0.16.0.8-1) (build 17.0.16+8-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-17.0.16.0.8-1) (build 17.0.16+8-LTS, mixed mode, sharing)

Prod 7.1.9 -> java -version

openjdk version "1.8.0_462"
OpenJDK Runtime Environment (build 1.8.0_462-b08)
OpenJDK 64-Bit Server VM (build 25.462-b08, mixed mode)

https://supportmatrix.cloudera.com/

https://docs.cloudera.com/cdp-private-cloud-base/7.3.1/cdp-private-cloud-base-installation/topics/cdpdc-java-requirements.html

From cloudera docs: If you are using JDK 17 on your cluster, you must add the following JVM options to the service:

--add-opens=java.base/java.lang=ALL-UNNAMED 
--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED 
--add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED 
--add-exports=java.base/sun.net.dns=ALL-UNNAMED 
--add-exports=java.base/sun.net.util=ALL-UNNAMED

to ensure the jobs run successfully.

any guidance on this might the issue for job failures.