Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

INSERT OVERWRITE with HIVE format is replacing old data with new?

New Contributor

I am having two table in hive and doing INSERT OVERWRITE one table into other table. Target table is an EXTERNAL table other is an INTERNAL Table. Both tables are in HIVE format.

SPARK.SQL(s" INSERT OVERWRITE TABLE VW_HISTORY1 " +
    s" SELECT NAMED_STRUCT('IN_ID', STG.IN_ID,...) AS key, "+
    s" STG_MEMKEY, STG.IN_OB_ID,..... from VW_HISTORY_UPDATE1 stg" )

 It is adding all recodes from VM_HISTORY_UPDATE1 to VW_HISTORY1.

 

And I want do same in azure data bricks. so created to tables as target is EXTERNAL and other is INTERNAL and did same. But here it is completely removing previous data and adding new data into it.

 

How can I do this to add records.

1 REPLY 1

Super Collaborator

check TBLPROPERTIES if transctional=ttrue, if yes, insert overwrite would work

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.