Created 03-14-2016 09:03 PM
I am developing a Java program that uses the Apache Sqoop Metastore API to complete full and incremental loads. I would like to store job details such as the check column in the job. Is there an example leveraging this API to illustrate?
Created 03-25-2016 06:42 AM
This should do it:
/** * If this is an incremental import then we save the user's state back to the metastore */ private void saveIncrementalState(SqoopOptions options) throws IOException { if (!isIncremental(options)) { return; } Map<String,String> descriptor=options.getStorageDescriptor(); String jobName=options.getJobName(); if (null != jobName && null != descriptor) { LOG.info("Saving incremental import state to the metastore"); JobStorageFactory ssf=new JobStorageFactory(options.getConf()); JobStorage storage=ssf.getJobStorage(descriptor); storage.open(descriptor); try { JobData data=new JobData(options.getParent(),this); storage.update(jobName,data); LOG.info("Updated data for job: " + jobName); } finally { storage.close(); } } }
Created 03-25-2016 06:42 AM
This should do it:
/** * If this is an incremental import then we save the user's state back to the metastore */ private void saveIncrementalState(SqoopOptions options) throws IOException { if (!isIncremental(options)) { return; } Map<String,String> descriptor=options.getStorageDescriptor(); String jobName=options.getJobName(); if (null != jobName && null != descriptor) { LOG.info("Saving incremental import state to the metastore"); JobStorageFactory ssf=new JobStorageFactory(options.getConf()); JobStorage storage=ssf.getJobStorage(descriptor); storage.open(descriptor); try { JobData data=new JobData(options.getParent(),this); storage.update(jobName,data); LOG.info("Updated data for job: " + jobName); } finally { storage.close(); } } }