Skip to main content
hayder-cardano
Gold ⭐️
Feb 2, 2023 2:33 PM UTC
Solved

iceberg load data with hive

hi

I want to know if its possible to use the LOAD DATA INPATH to load parquet files into an external iceberg table? 

right now I am getting the following error: 
Error while compiling statement: FAILED: SemanticException [Error 10101]: A non-native table cannot be used as target for LOAD   

 
does hive support LOAD DATA INPATH with Iceberg tables?
hayder-cardano
Gold ⭐️
Feb 6, 2023 2:07 PM UTC

going to answer myself. 

as of hive3, its not yet possible to directly bulk load external datasets (e.g. parquet / orc) files. so its necessary to do the following. 
create temporary managed hive table
LOAD DATA INPATH the parquet / orc / csv into the temporary managed hive table
create external table as iceberg. 
Select * and copy into the external iceberg table



this has worked with the following iceberg features. 
write, append, time-travel

1 reply

hayder-cardano
hayder-cardanoAuthorAnswer
Gold ⭐️
Feb 6, 2023 2:07 PM UTC

going to answer myself. 

as of hive3, its not yet possible to directly bulk load external datasets (e.g. parquet / orc) files. so its necessary to do the following. 
create temporary managed hive table
LOAD DATA INPATH the parquet / orc / csv into the temporary managed hive table
create external table as iceberg. 
Select * and copy into the external iceberg table



this has worked with the following iceberg features. 
write, append, time-travel

Translate