Support Questions

Find answers, ask questions, and share your expertise

how to load only new records from HIVE table

avatar
New Contributor

Hi every one

  I have table in HIVE, that consists product_name, product_id, total sales (only three columns) 

Table name : total_sales dataset

dataset:

yesterday sales:

product_ name

Product_ id

total sales

1000

laptop

500

1001

mobile

5000

1002

hard drive

550

Note:

With yesterday datasets (3 records) i can calculate final stock list in spark.  

spark :

product name

Product id

initial stock

total sales

Final stock

1000

laptop

1000

500

500

1001

mobile

15000

5000

10000

1002

hard drive

1000

550

450

 Today sales

product_ name

Product_ id

total sales

1000

laptop

100

1001

mobile

500

1002

hard drive

200

 note:

     Three more records get added in total_sales dataset (HIVE). 

total_sales table:

product name

Product id

total sales

1000

laptop

500

1001

mobile

5000

1002

hard drive

550

1000

laptop

100

1001

mobile

500

1002

hard drive

200

 

Question:

  1. How to check any new records get added or not in total_sales dataset.
  2. If new dataset gets added, how to export only new records?

Can anyone help me on this please

 

Thanks

sen

1 REPLY 1

avatar
Community Manager

Welcome to the community @sencae 

While you wait for a more knowledgable person to respond, I did find this older post that hopefully gets you closer to where you need to be. 

 

https://community.cloudera.com/t5/Support-Questions/How-to-retrieve-Latest-Uploaded-records-from-Hiv... 


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.