@sivasaravanakumar kAttached is the full example, and here are the highlights. Table in Mysql defined below. For best results use timestamp as your date/time field. If you use just "date" like in your table you are ending up with low time granularity, so if you run the same job more than once a day it will import all new records updated that day.
create table st1(id int, name varchar(16), ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP);
Populate the table with 5000 entries. Create and run a new Sqoop job writing into a hdfs directory, please adjust for hbase. I'm showing only the important output lines, see the attachment for full ouput (the "driver" option is required on the sandbox, you can ignore it, and I'm using only 1 mapper because my table is small):
[it1@sandbox ~]$ sqoop job --create incjob -- import --connect jdbc:mysql://localhost:3306/test --driver com.mysql.jdbc.Driver --username it1 --password hadoop --table st1 --incremental lastmodified -check-column ts --target-dir sqin -m 1 --merge-key id
[it1@sandbox ~]$ sqoop job --exec incjob
16/01/24 00:27:59 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.3.2.0-2950
16/01/24 00:28:09 INFO tool.ImportTool: Incremental import based on column ts
16/01/24 00:28:09 INFO tool.ImportTool: Upper bound value: '2016-01-24 00:28:09.0'
16/01/24 00:28:31 INFO mapreduce.ImportJobBase: Retrieved 5000 records.
16/01/24 00:28:31 INFO tool.ImportTool: Saving incremental import state to the metastore
16/01/24 00:28:31 INFO tool.ImportTool: Updated data for job: incjob
The first time all 5000 entries are imported. Note that import tool sets the "Upper bound value" of ts to the current time when the command is executed. Now, change 200 entries, and run the same job again:
[it1@sandbox ~]$ sqoop job --exec incjob
16/01/24 00:35:59 INFO tool.ImportTool: Incremental import based on column ts
16/01/24 00:35:59 INFO tool.ImportTool: Lower bound value: '2016-01-24 00:28:09.0'
16/01/24 00:35:59 INFO tool.ImportTool: Upper bound value: '2016-01-24 00:35:59.0'
16/01/24 00:36:20 INFO mapreduce.ImportJobBase: Retrieved 200 records.
16/01/24 00:36:57 INFO tool.ImportTool: Saving incremental import state to the metastore
16/01/24 00:36:58 INFO tool.ImportTool: Updated data for job: incjob
Now only 200 entries are imported. Lower bound value is the one set the first time, and the Upper bound value is updated to the current time, and so the job is ready for the next run. That's all, happy sqooping!
