I am pulling data from DB2 and storing in .avro format in HDFS.
After that through nifi triggering spark sql script to compare today's
and yesterday's avro file and insert newly added or updated records to
hive table. There might be records that gets deleted from DB2.
Now the records which gets deleted from DB2 that records need to mark as deleted in Hive table.
So now I am using Merge command to update hive table because I have
composite primary key. So when I use merge command to update table then
next day while inserting data to hive merged table then getting un-usual
results.
Is there any issus with merge command in Hive.
Hive Version:: Hive 1.2
Hadoop Version:: Hadoop 2.7.3.2.6.3.0-235
