- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
iceberg load data with hive
Created 02-02-2023 06:33 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi
I want to know if its possible to use the LOAD DATA INPATH to load parquet files into an external iceberg table?
right now I am getting the following error:
Error while compiling statement: FAILED: SemanticException [Error 10101]: A non-native table cannot be used as target for LOAD
Created 02-06-2023 06:07 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
going to answer myself.
as of hive3, its not yet possible to directly bulk load external datasets (e.g. parquet / orc) files. so its necessary to do the following.
create temporary managed hive table
LOAD DATA INPATH the parquet / orc / csv into the temporary managed hive table
create external table as iceberg.
Select * and copy into the external iceberg table
this has worked with the following iceberg features.
write, append, time-travel
Created 02-06-2023 06:07 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
going to answer myself.
as of hive3, its not yet possible to directly bulk load external datasets (e.g. parquet / orc) files. so its necessary to do the following.
create temporary managed hive table
LOAD DATA INPATH the parquet / orc / csv into the temporary managed hive table
create external table as iceberg.
Select * and copy into the external iceberg table
this has worked with the following iceberg features.
write, append, time-travel
