Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

INSERT OVERWRITE with HIVE format is replacing old data with new?

avatar
New Contributor

I am having two table in hive and doing INSERT OVERWRITE one table into other table. Target table is an EXTERNAL table other is an INTERNAL Table. Both tables are in HIVE format.

SPARK.SQL(s" INSERT OVERWRITE TABLE VW_HISTORY1 " +
    s" SELECT NAMED_STRUCT('IN_ID', STG.IN_ID,...) AS key, "+
    s" STG_MEMKEY, STG.IN_OB_ID,..... from VW_HISTORY_UPDATE1 stg" )

 It is adding all recodes from VM_HISTORY_UPDATE1 to VW_HISTORY1.

 

And I want do same in azure data bricks. so created to tables as target is EXTERNAL and other is INTERNAL and did same. But here it is completely removing previous data and adding new data into it.

 

How can I do this to add records.

1 REPLY 1

avatar
Guru

check TBLPROPERTIES if transctional=ttrue, if yes, insert overwrite would work