Support Questions
Find answers, ask questions, and share your expertise

Ingest data incrementally using Nifi from DB2

Highlighted

Re: Ingest data incrementally using Nifi from DB2

Super Guru

Your ExecuteSQL appears to have a query specified in the "SQL select query" property. This means it will ignore the contents of the flow files, which are each SQL statements themselves. Clear the "SQL select query" value, and then ExecuteSQL will use the flow file contents to execute those statements, which should have the SQL for incremental fetches.

Re: Ingest data incrementally using Nifi from DB2

Explorer

from Generatetablefetch I'm able to get the flow file . Upon running the flow, the select query failed to execute on DB2. On investigating we found that the query generated by GenerateTableFetch looked like this

select userid,timestamp from user11 where timestamp<='01-01-2018 12:00:00' order by timestamp limit 10000

And I have used the Nifi Expression language as per the https://community.hortonworks.com/articles/167733/using-generatetablefetch-against-db2.html and created a query like this

select ${generatetablefetch.columnnames} from ${generatetablefetch.tablename} where ${generatefetchtable.whereClause} order by ${generatetablefetch.maxColumnNames}  fetch first ${generatetablefetch.limit} rows only
select userid,timestamp from user11 where timestamp >= '01-01-2018 12:00:00' order by timestamp limit 1000


but i'm getting 


select userid,timestamp from user11 where order by timestamp limit 1000

In the above example where condition is not taking the value. please refer the screenshots for my configuration80483-nififlow.png

I think I have made halfway through this and stuck here .What is missing in this .