Support Questions
Find answers, ask questions, and share your expertise

DYNAMIC PARTITIONING with INCREMENTAL LOAD

DYNAMIC PARTITIONING with INCREMENTAL LOAD

Master Collaborator

here is the scenario :

I want to partition the data on PARTCOL and also want to do incremental load based on DATECOL .

1- I do a full load of the table using hcatalog cause I want to use dynamic portioning of data (which flags I would use to also create the hive table) ?.

2- now when I do a sqoop incremental load ..how does it know where to start with ? how will it find out the max(DATECOL) value?

can someone put down the general steps on how I can achive this ?