Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to stop QueryDatabaseTable after processing all the data?

Solved Go to solution
Highlighted

How to stop QueryDatabaseTable after processing all the data?

I am trying move a table from Oracle to HDFS. I used QueryDatabaseTable -> putHdfs processors and configured them. I can see data coming to hdfs.. but the process is running continuously and records are being added again and again. Am I doing anything wrong or missing something?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

How did you schedule QueryDatabaseTable?

If you didn't change anything on the scheduling tab of the processor, then the run schedule is 0 seconds which means as fast as possible. You most likely want to run this on some kind of timer or cron scheduling.

View solution in original post

8 REPLIES 8
Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

How did you schedule QueryDatabaseTable?

If you didn't change anything on the scheduling tab of the processor, then the run schedule is 0 seconds which means as fast as possible. You most likely want to run this on some kind of timer or cron scheduling.

View solution in original post

Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

I did not change anything on scheduling tab as I wanted it to run asap. The problem is that the processor is running continuously. So If I have 10 records in actual table, I see the count on hdfs keep on increasing (dumping these 10 again and again). I expect it to stop after moving just these initial 10 records onto hdfs.

Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

If you use timer scheduling it will still execute right away, so if you set 30 seconds it will run right away then wait 30 seconds before running again.

Can you provide all of the configuration you entered for the processor?

You would need to provide the "Maximum-Value Columns" in order for it to track where it left off and pick up there on next execution.

Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

Thanks you for the information. So, if I set "Run Schedule" to default i.e ) sec, It will run tasks one after other again and again. And If I want it to execute only once, I would make it to some huge value or use event or CRON timer.

Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

Correct. If you truly only want to run it once, then make the timer schedule larger and just manually start and stop the processor.

Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

New Contributor
@Bryan Bende

Hi Bryan,

Small question regarding your earlier reply.

In my scenario, the processor should run once when i started and it should run again when there is any update. I'm not sure that when the Database will get updated. So, i can't use Timer here. I have tried by specifying the "Maximun-value-columns", but no luck.

Can you please help me in finding the way to do this.

Thanks,

Srikanth.

Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

Super Guru

What does your table look like? Is there a column that is guaranteed to be "strictly increasing" for each added/updated row? Sometimes this is the ID column (if using an autoincrementing integer that doesn't roll over), or perhaps a timestamp column such as "Last Updated". If you have no such column, then you will want to follow Bryan's advice on scheduling and start/stop.

Highlighted

Re: How to stop QueryDatabaseTable after processing all the data?

New Contributor

Thanks. Its worked after giving the schedule time.

Don't have an account?
Coming from Hortonworks? Activate your account here