@enam  Looks like you have a bad file filter regex in your listSFTP processor configuration. .*file.*\.xls Above looks for any character for 1 or more characters until is finds the last occurrence of string "file" followed by any character for as many characters until last occurrence of string ".xls".  However, all your filenames start with "file" and have no characters before it. Try modifying your file filter regex by removing the ".*" before "file": file.*\.xls  Right click on processor, select "view state", and then "clear state".  Then start the listSFTP processor again to see if generated NiFi FlowFiles for each file on your SFTP server. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Anderosn  Is your InvokeHTTP processor triggered by a FlowFile from an inbound connection to the processor or does it have no inbound connections and executes purely based on configured run schedule? This is one of very few processors where an inbound connection is optional, but behavior is different dependent on the configuration chosen.   With no inbound connection there is no FlowFile to "retry" when you encounter "failure" or "No retry" result from execution.  Because really it is retrying every time it executes essentially with no inbound connection. You could use a GenerateFlowFile processor to feed an empty trigger FlowFile to the invokeHTTP processor to trigger its execution.  This would then give you a FlowFile that Retry configuration can use. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@glad1  Based on what you shared, you may be able to accomplish what you are trying to do using a couple additional processors and a Child Process Group that utilizes "Process Group FlowFile Concurrency" settings. So your use case involves: For each 1 FlowFile output by your ExecuteSQLRecord processor you want to do ALL the following before the next FlowFile produced by ExecuteSQL record is processed: 1. Split the FlowFIle into X number of FlowFiles using SplitJson. 2. Modify each one of the produced Split FlowFiles using UpdateRecord. 3. Write all those modified FlowFiles ot another Database using PutDatabaseRecord 4. ExecuteSQL only once to update Record that all splits were processed. Then repeat above for next produce FlowFile If this is correct, here is what you might want to try: 1. create a Process Group that you will insert in teh middle of this flow as shown in the following image: 2. Configure that Process group as follows: Important properties to set here are: - Process Group FlowFile Concurrency = Single FlowFile Per Node - Process Group Outbound Policy = Batch Output What this does is allow only one FlowFile (per node in multi-node NiFi) to enter this PG at any given time. Inside this Process group you will have handle the processing of this FlowFile (split, update, putDB).  The outbound policy will not release any of the Produced Splits from teh PG until all are queued at the output port. You'll notice I added one additional optional processor ModifyBytes to your dataflow (configured with "Remove all Content = true).  This will zero out the content on the FlowFiles after they were written using the PutDatabaseRecord processor.  Then those FlowFiles with no content now are sent to connection feeding output port where they are held until all splits produced are queued.  They will then all be released at once from the PG to the next new processor MergeContent. The MergeContent processor will merge all those FlowFiles into a single FlowFile that feeds your ExecuteSQL (UpdateRecordStatus toP) processor.  Now you have a single notification for the original FlowFile that was split and processed.  Additionally you have created separation between each source FlowFile processed at start of dataflow. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@FrankHaha  Have you tried using the "Infer Schema" Schema Access Strategy in the JsonTreeReader 1.24.0 controller services instead of fetching schema from AvroSchemaRegistry?    Another option would be to use the ExtractRecordSchema 1.24.0 processor along with JsonTreeReader 1.24.0 controller services configured with "Infer Schema" Schema Access Strategy" to output the schema into a FlowFile Attribute "avro.schema".  You can then get the produced schema from that FlowFile Attribute to add to your AvroSchemaRegsitry for future use. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more