Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Prevent sqoop import

Highlighted

Prevent sqoop import

Explorer

I don't want to do a sqoop import if RDMS table has zero records. Anyway i can do this without using sqoop eval.

4 REPLIES 4
Highlighted

Re: Prevent sqoop import

Super Guru

@vasanth rajendran Can you please elaborate on what approach you would like to take if not using sqoop-eval tool? Do you think a certain approach if available will do a better job of checking if there any new records?

One way to bypass running Sqoop is to create a trigger on your Table. When data is inserted, this trigger can execute a Sqoop script which moves data from RDBMS to Hadoop.

Highlighted

Re: Prevent sqoop import

Explorer

I'm asking is there any alternative to sqoop eval. My requirement is, If RDBMS table has zero records i should not run the sqoop job. I need to do a validation check, Is record exists in source table (RDBMS). If it has zero records i need to skip the sqoop import process.

Highlighted

Re: Prevent sqoop import

Super Guru

@vasanth rajendran

And how would you check if records exist in source table? that is exactly what sqoop-eval will do. Run a Sqoop eval, if there are records, then run sqoop import.

If you don't want to run sqoop eval, then use jdbc driver to get row count from the table. If it's greater than zero, run your sqoop import.

Highlighted

Re: Prevent sqoop import

Explorer

Thanks. Will use sqoop eval.

Don't have an account?
Coming from Hortonworks? Activate your account here