question Loading data into Hive Table from HDFS deletes the file from source directory(HDFS). in Support Questions

Loading data into Hive Table from HDFS deletes the file from source directory(HDFS).

ravikumashi — Sat, 23 Jul 2016 20:03:44 GMT

Hi All,

When we Load data into Hive table from HDFS, it deletes the file from source directory(HDFS) is there a way we can keep the file in the source directory and load the data into hive table as well.

I used the below query;

LOAD DATA INPATH 'source_file_path' OVERWRITE INTO TABLE TABLENAME;

Re: Loading data into Hive Table from HDFS deletes the file from source directory(HDFS).

jyadav — Sat, 23 Jul 2016 20:11:12 GMT

@Ravikumar Kumashi

If you don't want to loss the source data copy while loading then the best way would be to create external table over that existing hdfs directory OR you can also make a copy of your source directory and create an external hive table that should point to new dir location.

hadoop fs -cp /path/old/hivetable /path/new/hivetable
create external table table_name ( id int, myfields string )
 location '/path/new/hivetable';

Re: Loading data into Hive Table from HDFS deletes the file from source directory(HDFS).

ravikumashi — Sat, 23 Jul 2016 20:16:48 GMT

@Jitendra Yadav

Thank you!!! that works for me.

I thought there is a way to keep the file in source directory and load the data into managed table as well and looks like there is no way for that.

Re: Loading data into Hive Table from HDFS deletes the file from source directory(HDFS).

aza — Wed, 30 Sep 2020 09:54:20 GMT

In my case, source file gets removed, when I load a single file with 'OVERWRITE' clause.

files stay when I load without 'OVERWRITE' clause for a set of files with a pattern (say _*.txt)