How to display the delta changes for a db using nifi

[#1]

I have a requirement like, I will be using mysql database and cassandra database.So I should be able to read data from both the databases based on hourly,weekly,monthly or specified date range. When we read the table from database it should display the delta changes for that table. So which nifi processors I should use to show delta changes for both mysql and cassandra db

[#2]

You can use "QueryDatabaseTable" as input processor, but you can´t use it twice in one flow (Im searching for a solution that I can get a delta in one flow for 2 Fields btw. :D).

You have to set the Property "Maximum-value Columns". It can be a date, id or something else to have the maximum of a value.

At "view state" you can see the value of "Maximum-value Columns" field you set.

To set up the runtime you can go to "Schedule" at the processor and change the "Run Schedule"

If you need more input, please let me know

[#3]

Thanks @Faerballert , as you suggested I did the following 

Set the Maximum-Value-Columns with modified_time, later I checked the View state for the modified_time column and able to see the modified column which I updated.

I want to know How I can get the changed data over the period of time, for example if I have modified the column in last 1hr/1 day/1 week then I should be able to see all the modified data.

[#4]

Go to the Processor and select "Scheduling".

As Scheduling strategy you can use "Cron driven"

and at the run schedule there you can set, when the Processor should be check for a Delta.

Examples:

Every 2 hours

0 0 0/2 1/1 * ? *

Daily at 1 a.m.

0 0 1 1/1 * ? *

Weekly at 3 p.m. on every monday

0 0 15 ? * MON *

[#5]

Hi @Faerballert ,

How can I achieve the same delta changes on cassandra. could u please share some processors which I can use and if I'm using QueryDatabaseTable its asking for jdbc driver and jar details. So please share your thought

[#6]

1. Download HDP Sandbox
2. MySQL database (Should already be present in the sandbox)
3. NiFi 0.6 or later (Download and install a new version of NIFI or use Ambari to install NIFI in the sandbox)

Use this code:

insert into cdc_test values (3, 'cdc3', null, null);

insert into cdc_test values (4, 'cdc3', null, null);

insert into cdc_test values (5, 'cdc3', null, null);

insert into cdc_test values (6, 'cdc3', null, null);

insert into cdc_test values (7, 'cdc3', null, null);

insert into cdc_test values (8, 'cdc3', null, null);

insert into cdc_test values (9, 'cdc3', null, null);

insert into cdc_test values (10, 'cdc3', null, null);

insert into cdc_test values (11, 'cdc3', null, null);

insert into cdc_test values (12, 'cdc3', null, null);

insert into cdc_test values (13, 'cdc3', null, null);

I hope this helps!

Regards,

Lewis