Support Questions
Find answers, ask questions, and share your expertise

how to schedule and maintain incremental sqoop jobs for 1600 tables


Hi All,

We have 1600 Rdbms tables to be imported into Hive using sqoop incremental import jobs. But, when I think of creating that many sqoop jobs and scheduling them in oozie or shell script, it seems challenging . I feel that it will be hard to maintain and find which job or script is for which table. has anyone dealt with such a scenario ? Please suggest how to schedule sqoop incremental jobs periodically which can be easily maintained and checked in case of any issues. Thanks!