Support Questions
Find answers, ask questions, and share your expertise

snapshots of tables by date

Expert Contributor

we are using hive with hdfs and often need to know what a table looked like on a particular date.

so, we have directories like these:

/user/abc/ReportSnaps/tableName/{date}

whenever we need to know what a table looked like on a particular date,

we change the location of table, map it to location for that date and get the data.

Now, we need to have reports like:

date | columnFromReportForTheDay   

How do we achieve this? A lot of these tables we are taking snapshots for do not have a date column in them and are just truncated everyday and updated with new data.

my question is let's say there is a column percentage in this table.

I want to be able to have a final table:

date               percentage

20170808           50%

20170809           40%

from files that are stored like this:

drwxr-xr-x   - hue   hue          0 2017-08-08 18:32 /user/hue/oostablecatalogue/20170808 
drwxr-xr-x   - tarun hue          0 2017-08-09 10:22 /user/hue/oostablecatalogue/20170809
2 REPLIES 2

Explorer

Hi, If you run "ls -ltr" command on each of this file or table names , it will return modified date , you can use this in reporting.Hope this helps. Regards, Fahim

Expert Contributor

I don't think you understood the problem here. Of course, I can get the dates,

my question is let's say there is a column percentage in this table.

I want to be able to have a final table:

date percentage

20170808 50%

20170809 40%

from files that are stored like this:

drwxr-xr-x   - hue   hue          0 2017-08-08 18:32 /user/hue/oostablecatalogue/20170808 
drwxr-xr-x   - tarun hue          0 2017-08-09 10:22 /user/hue/oostablecatalogue/20170809