I am new to NiFi and can use some help in designing a flow. I need to run a SQL query against a database table, dump output of SQL query into a csv file with fixed header,SFTP the file and then run an update SQL against source table to indicate that records have been processed. ExecuteSQL will run every couple of hours.
I am think that flow would look something like:
ExecuteSQL -> ConvertRecord ( Avro to Free form text) -> Some logic to generate Header -> MergeContent -> PutFile ( to generate CSV file) -> PutSFTP --> PutSQL
Q1. How can I generate csv file with Header ?
Q2. Any chances of above flow picking records in loop. For instance, ExecuteSQL picks up same set of records multiple times before PutSQL marks records as "already processed"