Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Solved
Go to solution
How to create/update Sqoop Jobs in Java?
Labels:
- Labels:
-
Apache Sqoop
Contributor
Created ‎03-14-2016 09:03 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am developing a Java program that uses the Apache Sqoop Metastore API to complete full and incremental loads. I would like to store job details such as the check column in the job. Is there an example leveraging this API to illustrate?
1 ACCEPTED SOLUTION
Guru
Created ‎03-25-2016 06:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This should do it:
/** * If this is an incremental import then we save the user's state back to the metastore */ private void saveIncrementalState(SqoopOptions options) throws IOException { if (!isIncremental(options)) { return; } Map<String,String> descriptor=options.getStorageDescriptor(); String jobName=options.getJobName(); if (null != jobName && null != descriptor) { LOG.info("Saving incremental import state to the metastore"); JobStorageFactory ssf=new JobStorageFactory(options.getConf()); JobStorage storage=ssf.getJobStorage(descriptor); storage.open(descriptor); try { JobData data=new JobData(options.getParent(),this); storage.update(jobName,data); LOG.info("Updated data for job: " + jobName); } finally { storage.close(); } } }
1 REPLY 1
Guru
Created ‎03-25-2016 06:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This should do it:
/** * If this is an incremental import then we save the user's state back to the metastore */ private void saveIncrementalState(SqoopOptions options) throws IOException { if (!isIncremental(options)) { return; } Map<String,String> descriptor=options.getStorageDescriptor(); String jobName=options.getJobName(); if (null != jobName && null != descriptor) { LOG.info("Saving incremental import state to the metastore"); JobStorageFactory ssf=new JobStorageFactory(options.getConf()); JobStorage storage=ssf.getJobStorage(descriptor); storage.open(descriptor); try { JobData data=new JobData(options.getParent(),this); storage.update(jobName,data); LOG.info("Updated data for job: " + jobName); } finally { storage.close(); } } }
