Reply
Highlighted
Explorer
Posts: 14
Registered: ‎02-08-2019

sqoop overwite data files ?

[ Edited ]
I imported data from RDBMS to HDFS through sqoop with the following command: 
sqoop import --connect ... --table ... --hive-import --hive-table --m 20 etc.
 
So this command will create 20 files under a dir in hdfs.

If I decide later to import another data and overwrite the existing one I will use the following command: 

sqoop import --connect ... --table ... --hive-import --hive-overwrite --hive-table --m 10.

 

Will Sqoop go to the dir and remove all the previous 20 files and create 10 new ones ?

Master
Posts: 430
Registered: ‎07-01-2015

Re: sqoop overwite data files ?

I think it will throw you an error saying that the destination directory is not empty. At least this is how I remember it. If not, then it will erase the directory definitely.
Explorer
Posts: 14
Registered: ‎02-08-2019

Re: sqoop overwite data files ?

[ Edited ]

@Tomas79  I had the chance to test it and I noticed that Sqoop removing all the pro-existing files and creates new ones.

Announcements
New solutions