Skip to main content
gurumoorthyk
gurumoorthyk
Bronze III
Jan 21, 2026 4:38 AM UTC
Solved

Assistance Required for Deploying Iceberg SerDe in CDP 7.1.9 Cluster

Hi Team,

We are planning to deploy the Iceberg Hive runtime JAR (iceberg-hive-runtime.jar) in our CDP Private Cloud Base 7.1.9 cluster (Cloudera Manager 7.11.3) to enable Hive and Spark to work with Iceberg tables.

Iceberg Hive runtime JAR: iceberg-hive-runtime-1.3.1.jar,iceberg-spark-runtime-3.3_2.12-1.3.1.jar

step: 1
Upload the JAR to HDFS

Create a directory in HDFS:
hdfs dfs -mkdir -p /user/share/lib


Upload the JAR:

hdfs dfs -put iceberg-hive-runtime.jar /user/share/lib/
hdfs dfs -chmod 755 /user/share/lib/iceberg-hive-runtime.jar


Configure Hive via Cloudera Manager

Navigate to Hive → Configuration → Auxiliary JARs Directory

Add:

hdfs:///user/share/lib


Step 2:
Configure Spark via Cloudera Manager

Navigate to Spark → Configuration → Spark Client Advanced Configuration Snippet (Safety Valve)

Add:
spark.yarn.jars hdfs:///user/share/lib/iceberg-hive-runtime.jar

Step 3:
Restart Services
Restart Hive, Spark, and YARN.

 

We would like guidance and confirmation on the steps, any possible impacts, and best practices to ensure a smooth deployment.

  • Are the steps outlined above correct and fully supported for CDP 7.1.9?
  • Are there any additional recommendations we should be aware of?
  • Can you provide guidance on the best way to validate the deployment?

Environment Details:

  • CDP Private Cloud Base: 7.1.9
  • Cloudera Manager: 7.11.3
  • Hadoop Cluster (metadata connection)
  • Keytab authentication
  • Spark Universal enabled
smruti
smrutiAnswer
Gold ⭐️⭐️⭐️
Jan 21, 2026 9:59 AM UTC

@gurumoorthyk Iceberg is not supported with Hive in CDP on-prem 7.1.9. You may enable it using the runtime jars, but some functionalities may not work, and at Cloudera, we do not support it. Iceberg with Hive is supported from 7.3.1(on prem).

The procedure you have listed to load iceberg runtime jar to Hive classpath is correct.

View in thread

4 replies

VidyaSargur
Community Manager
VidyaSargur
Community Manager
Jan 21, 2026 7:44 AM UTC

@gurumoorthyk Welcome to our community! To help you get the best possible answer, I have tagged in our Iceberg experts @smruti, @dipankartnt, and @tjangid , who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
smruti
Gold ⭐️⭐️⭐️
smrutiAnswer
Gold ⭐️⭐️⭐️
Jan 21, 2026 9:59 AM UTC

@gurumoorthyk Iceberg is not supported with Hive in CDP on-prem 7.1.9. You may enable it using the runtime jars, but some functionalities may not work, and at Cloudera, we do not support it. Iceberg with Hive is supported from 7.3.1(on prem).

The procedure you have listed to load iceberg runtime jar to Hive classpath is correct.

Translate
aranjireddy
Gold ⭐️
aranjireddy
Gold ⭐️
Jan 21, 2026 5:35 PM UTC

@gurumoorthyk adding to Smruthi's update, Iceberg + Hive+ Spark2 is not supported in CDP 7.1.9, it's officially supported from CDP 7.3.1. We don't need to add any jars explicitly in 7.3.1. By default, service[Impala,spark3, Hive] classpaths will have the required iceberg jars.

In CDP 7.1.9, you can use Iceberg with Impala and Spark 3, but with limited functionalities. It does not require any classpath manipulation.

Doc: https://docs.cloudera.com/cdp-public-cloud/cloud/cdp-iceberg/topics/iceberg-in-cdp.html

FYI: [You can check the commands below in your env]

[root@host1 jars]# pwd [You can see impala & spark3 classpaths by default has the iceberg jars]
/opt/cloudera/parcels/
[root@ccycloud-1.nightly-71x-kl.root.comops.site jars]# find ../ -name *iceberg-hive-runtime*
../lib/oozie/oozie-sharelib-yarn/lib/sqoop/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar
../lib/hive/lib/atlas-hive-plugin-impl/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar
../lib/impala/lib/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar ====>impala classpath
../lib/atlas/hook/hive/atlas-hive-plugin-impl/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar

../jars/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar
[root@host1 jars]# find ../ -name *iceberg-spark*
find ../.. -name *iceberg-spark*
../../SPARK3-3.5.4.3.5.7191000.0-30-1.p0.68499982/lib/spark3/iceberg/iceberg-spark-runtime-3.5_2.12-1.3.0.3.5.7191000.0-30.jar.  =====>SPARK3 Classpath
../../CDH-7.1.9-1.cdh7.1.9.p1003.74501001/lib/oozie/oozie-sharelib-yarn/lib/spark3/iceberg-spark-runtime-3.3_2.12-1.3.0.3.3.7191000.1-138.jar
../../CDH-7.1.9-1.cdh7.1.9.p1003.74501001/jars/iceberg-spark-runtime-3.3_2.12-1.3.0.3.3.7191000.1-138.jar

Translate
cjervis
Community Manager
cjervis
Community Manager
Jan 30, 2026 3:22 PM UTC

@gurumoorthyk have you found the answer to your question. If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

Translate
Keep the questions coming
Terms & ConditionsAccessibility statement