Created 10-26-2017 03:29 PM
I need to schedule the job for very one hour
Created 10-26-2017 03:36 PM
I would use Sqoop incremental import. Have you tried it?
https://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_incremental_imports
Created 10-26-2017 03:50 PM
Do you have a working sqoop command?
With that information, you can create an hourly oozie job.
Start with a one-off workflow.xml file - find the documentation here.
https://oozie.apache.org/docs/4.2.0/DG_SqoopActionExtension.html
Make sure you can run the workflow before working on the coordinator.
Then you can make an hourly coordinator like this and put the "jobStart" and "jobEnd" properties in the oozie config file.
<coordinator-app name="DB2-Export" frequency="${coord:hours(1)}" start="${jobStart}" end="${jobEnd}" timezone="UTC" xmlns="uri:oozie:coordinator:0.2"> <controls> <concurrency>1</concurrency> <execution>FIFO</execution> <throttle>1</throttle> </controls> <action> <workflow> <app-path>${wf_application_path}</app-path> </workflow> </action> </coordinator-app>
You would execute this like
oozie job -config db2-export-cooord.properties -run
Where that property file might contain
jobTracker=namenode.fqdn:8050 nameNode=hdfs://hadoop_cluster wf_application_path=hdfs://path/to/db2-export/ oozie.coord.application.path=${wf_application_path} jobStart=2017-11-01T09:00Z jobEnd=2099-11-09T09:00Z