I am running into an issue where I am unable to successfully use a PutSQL or ExecuteSQL after returning from a Batch Processor group. The general scenario is this: I need to watch a folder for files that will be created with information to be uploaded to a PostgreSQL database. The files will have multiple lines, each line is to be inserted as a separate record into a temporary/staging table. When all the records from that file have been uploaded, I need to call a stored procedure that will process all those records and update other permanent tables all at once. I came across the concept of using a processor group set with 'Process Group Outbound Policy' set to 'Batch Output'. So, I am able to use GetFile to get each file as it is produced, then pass it into my batch processor group, where it splits the lines into individual flow files, and performs the inserts into my staging table. That all works perfectly. I then use a MergeContent processor to reduce those split files back into a single flow file, so I can subsequently use either ExecuteSQL or PutSQL to call my stored procedure - once. But I then receive an error stating, "Failed to update database for FlowFile[filename=MYFILE.TXT] due to The column index is out of range: 1, number of columns: 0.; it is possible that retrying the operation will succeed, so routing to retry: org.postgresql.util.PSQLException: The column index is out of range: 1, number of columns: 0." The strange thing is that it doesn't matter what SQL statement I am trying to use, whether a 'CALL' to my stored procedure, or a SELECT statement to any table, I get the exact same error. I also tried a test to immediately use a PutSQL to insert records to another test table, after returning from the process group, prior to the MergeContent processor, so that's not the issue either. Even more strange is that I can take the same SQL processor(s) and wire it into the flow prior to the batch process group, and it works just fine. I have other SQL-related processors at the main level, plus the PutSQL processor within the Processor Group that performs each individual insert, and they all work fine. And they all use the same JDBC Connection Pool. So what's the issue? Do I have something misconfigured? Is something not getting passed in and out of the Process Group? Or, considering I'm am using NiFi version 2.0.0-M4, is there a bug somewhere? Thanks.
... View more
