Support Questions

Find answers, ask questions, and share your expertise

iceberg load data with hive

avatar
New Contributor

hi

I want to know if its possible to use the LOAD DATA INPATH to load parquet files into an external iceberg table? 

right now I am getting the following error: 
Error while compiling statement: FAILED: SemanticException [Error 10101]: A non-native table cannot be used as target for LOAD   

 
does hive support LOAD DATA INPATH with Iceberg tables?
1 ACCEPTED SOLUTION

avatar
New Contributor

going to answer myself. 

as of hive3, its not yet possible to directly bulk load external datasets (e.g. parquet / orc) files. so its necessary to do the following. 
create temporary managed hive table
LOAD DATA INPATH the parquet / orc / csv into the temporary managed hive table
create external table as iceberg. 
Select * and copy into the external iceberg table



this has worked with the following iceberg features. 
write, append, time-travel

View solution in original post

1 REPLY 1

avatar
New Contributor

going to answer myself. 

as of hive3, its not yet possible to directly bulk load external datasets (e.g. parquet / orc) files. so its necessary to do the following. 
create temporary managed hive table
LOAD DATA INPATH the parquet / orc / csv into the temporary managed hive table
create external table as iceberg. 
Select * and copy into the external iceberg table



this has worked with the following iceberg features. 
write, append, time-travel