How to create/update Sqoop Jobs in Java?

ltsimps1 — Tue, 15 Mar 2016 04:03:20 GMT

I am developing a Java program that uses the Apache Sqoop Metastore API to complete full and incremental loads. I would like to store job details such as the check column in the job. Is there an example leveraging this API to illustrate?

Re: How to create/update Sqoop Jobs in Java?

egarelnabi — Fri, 25 Mar 2016 13:42:06 GMT

This should do it:

/** 
 * If this is an incremental import then we save the user's state back to the metastore
*/

private void saveIncrementalState(SqoopOptions options) throws IOException {
  if (!isIncremental(options)) {
    return;
  }
  Map<String,String> descriptor=options.getStorageDescriptor();
  String jobName=options.getJobName();
  if (null != jobName && null != descriptor) {
    LOG.info("Saving incremental import state to the metastore");
    JobStorageFactory ssf=new JobStorageFactory(options.getConf());
    JobStorage storage=ssf.getJobStorage(descriptor);
    storage.open(descriptor);
    try {
      JobData data=new JobData(options.getParent(),this);
      storage.update(jobName,data);
      LOG.info("Updated data for job: " + jobName);
    }
  finally {
      storage.close();
    }
  }

}

question Re: How to create/update Sqoop Jobs in Java? in Archives of Support Questions (Read Only)

How to create/update Sqoop Jobs in Java?

Re: How to create/update Sqoop Jobs in Java?