Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to load only new records from HIVE table

avatar
New Contributor

Hi every one

  I have table in HIVE, that consists product_name, product_id, total sales (only three columns) 

Table name : total_sales dataset

dataset:

yesterday sales:

product_ name

Product_ id

total sales

1000

laptop

500

1001

mobile

5000

1002

hard drive

550

Note:

With yesterday datasets (3 records) i can calculate final stock list in spark.  

spark :

product name

Product id

initial stock

total sales

Final stock

1000

laptop

1000

500

500

1001

mobile

15000

5000

10000

1002

hard drive

1000

550

450

 Today sales

product_ name

Product_ id

total sales

1000

laptop

100

1001

mobile

500

1002

hard drive

200

 note:

     Three more records get added in total_sales dataset (HIVE). 

total_sales table:

product name

Product id

total sales

1000

laptop

500

1001

mobile

5000

1002

hard drive

550

1000

laptop

100

1001

mobile

500

1002

hard drive

200

 

Question:

  1. How to check any new records get added or not in total_sales dataset.
  2. If new dataset gets added, how to export only new records?

Can anyone help me on this please

 

Thanks

sen

1 REPLY 1

avatar
Community Manager

Welcome to the community @sencae 

While you wait for a more knowledgable person to respond, I did find this older post that hopefully gets you closer to where you need to be. 

 

https://community.cloudera.com/t5/Support-Questions/How-to-retrieve-Latest-Uploaded-records-from-Hiv... 


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.