Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Insert into Iceberg with TEZ works on CDP but not on local environment

avatar
New Contributor

Hello

I want to set up a local environment of Hive with Iceberg similar to my working environment - CDP public cloud 7.2.15 so I can test locally.

The problem is that on my local environment insert into iceberg tables (via Hive) does not work with Tez execution engine (works only with MapReduce). And according to the Iceberg documentation it should not work with Tez for Hive 3. But the thing is that it does work on CDP - I am able to insert data into Iceberg tables with Tez on Hive 3.

Is there a way to make it work locally? Thank you

 

Here is my local config:

Hadoop: 3.2.1

Hive: 3.1.3 (on CDP 3.1.3000.2023.0.14.0-155 is used)

Iceberg-hive-runtime: 0.13.1.7.2.15.0-147 (the same version is used on CDP)

Tez: 0.9.1.7.2.15.0-147 (the same version is used on CDP)

2 REPLIES 2

avatar
Expert Contributor

Hi,

You can try to use CDP public cloud 7.2.17 which officially supports iceberg on Hive.

https://docs.cloudera.com/runtime/7.2.17/iceberg-overview/topics/iceberg-overview.html

 

avatar
New Contributor

Hello @BennyZ 

Thanks for reply. I can use Iceberg on 7.2.15 as well.

My question was how come that insert into iceberg tables works on Tez execution engine when according to the documentation it should not (only with MapReduce)?