Created 03-14-2016 09:03 PM
I am developing a Java program that uses the Apache Sqoop Metastore API to complete full and incremental loads. I would like to store job details such as the check column in the job. Is there an example leveraging this API to illustrate?
Created 03-25-2016 06:42 AM
This should do it:
/**
* If this is an incremental import then we save the user's state back to the metastore
*/
private void saveIncrementalState(SqoopOptions options) throws IOException {
if (!isIncremental(options)) {
return;
}
Map<String,String> descriptor=options.getStorageDescriptor();
String jobName=options.getJobName();
if (null != jobName && null != descriptor) {
LOG.info("Saving incremental import state to the metastore");
JobStorageFactory ssf=new JobStorageFactory(options.getConf());
JobStorage storage=ssf.getJobStorage(descriptor);
storage.open(descriptor);
try {
JobData data=new JobData(options.getParent(),this);
storage.update(jobName,data);
LOG.info("Updated data for job: " + jobName);
}
finally {
storage.close();
}
}
}
Created 03-25-2016 06:42 AM
This should do it:
/**
* If this is an incremental import then we save the user's state back to the metastore
*/
private void saveIncrementalState(SqoopOptions options) throws IOException {
if (!isIncremental(options)) {
return;
}
Map<String,String> descriptor=options.getStorageDescriptor();
String jobName=options.getJobName();
if (null != jobName && null != descriptor) {
LOG.info("Saving incremental import state to the metastore");
JobStorageFactory ssf=new JobStorageFactory(options.getConf());
JobStorage storage=ssf.getJobStorage(descriptor);
storage.open(descriptor);
try {
JobData data=new JobData(options.getParent(),this);
storage.update(jobName,data);
LOG.info("Updated data for job: " + jobName);
}
finally {
storage.close();
}
}
}