Hi every one
I have table in HIVE, that consists product_name, product_id, total sales (only three columns)
Table name : total_sales dataset
dataset:
yesterday sales:
product_ name | Product_ id | total sales |
1000 | laptop | 500 |
1001 | mobile | 5000 |
1002 | hard drive | 550 |
Note:
With yesterday datasets (3 records) i can calculate final stock list in spark.
spark :
product name | Product id | initial stock | total sales | Final stock |
1000 | laptop | 1000 | 500 | 500 |
1001 | mobile | 15000 | 5000 | 10000 |
1002 | hard drive | 1000 | 550 | 450 |
Today sales
product_ name | Product_ id | total sales |
1000 | laptop | 100 |
1001 | mobile | 500 |
1002 | hard drive | 200 |
note:
Three more records get added in total_sales dataset (HIVE).
total_sales table:
product name | Product id | total sales |
1000 | laptop | 500 |
1001 | mobile | 5000 |
1002 | hard drive | 550 |
1000 | laptop | 100 |
1001 | mobile | 500 |
1002 | hard drive | 200 |
Question:
- How to check any new records get added or not in total_sales dataset.
- If new dataset gets added, how to export only new records?
Can anyone help me on this please
Thanks
sen