Hi i am wokring on it can you share me the flow file configuration and sp it will helpful to me   ... View more

I dont think you can use the same jolt spec for two different schema. Jolt spec doesnt allow you to use the same key multiple times. You have to know beforehand what schema are you getting then direct to different jolt transformation processor , or store the spec dynamically in an attribute to pass it to single jolt processor ... View more

@Kiranq, What I found is that the JoltTransformRecord expects only single record to work with hence the name. I noticed when I try to pass an array I was getting the error "...error transforming the first record", however if I pass just one json record it works. If you have an array of json\csv and you are looking to split and process each record individually then I would suggest that you split the records before the JoltTransformRecord. If you dont want to split the array then I recommend using JoltTranformJson first and then use Convert Record processor to convert to CSV. ... View more

@sarithe  You may also want to take a look at Process Group (PG) FlowFile Concurrency configuration options as a possible design path since there does not  appear to be any dependency between task 1 and task 2 in your description.  You just want to make sure that not more than 2 tasks are executing concurrently. You move your processors that handle the 2 task executions inside two different child PGs configured with "Single FlowFile per Node" Process Group FlowFile Concurrency.  Within the PG you create an input port and output port.  Between these two ports you handle your task dataflow.  Outside this PG (parent PG level), you handle the triggering FlowFiles.   The task PGs will allow 1 FlowFile at a time to enter that PG and because of the FlowFile Concurrency setting, not allow any more FlowFiles to enter this PG until that FlowFile processes out.     As you can see from above example, each task PG is only processing a single FlowFile at a time.  I built this example so that task 2 always takes longer, so you see that task 1 Pg is outputting more FlowFile processed the Task 2 PG while still making sure that on two tasks are ever being executed concurrently. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more

Thank you so much SAMSAL it's working  ... View more

@manishg  The ListFile does not pickup any files.  It simply generates a zero content NiFI FlowFile for each file found in the target directory.  That FlowFile only has metadata about the target content.  The FetchFile processor utilizes that metadata to fetch that actual content and add it to the FlowFile.  The value added here happens when you have a lot target files to ingest.  To avoid having all the disk I/o related to that content on one node, you can redistribute the zero byte FlowFiles across all nodes so that each node now in a distributed way fetches the content (This works assuming that same target directory is mounted on all NiFi cluster nodes).  As @SAMSAL shared you could use Process Group (PG) FlowFile concurrency to accomplish the processing of one FlowFile at a time. The ListFile will still continue to list all FlowFiles in target directory (writes state and continues to list new files as they get added to input directory).  You can then feed the outbound connection of your ListFile to a PG configured with "Single FlowFile Per Node" FlowFile concurrency.  This will prevent any other FlowFile queued between ListFile and the PG to enter the PG until the first FlowFile has processed through that PG.   So your first processor inside the PG would be your FetchFile processor.   Now if you were to configure Load Balanced Connection on that connection between ListFile and the PG, You would end up with each node in your NiFi cluster processing a single File at a time.  This gives you some concurrency if you want it.  However, if you have a strict one file at a time, you would not configure load balanced connection. Hope this helps, Matt ... View more

I dont think you can use PutSQL for this. Try ExecuteSQL or you might have to use ExecuteScript processor as other posts suggest: https://community.cloudera.com/t5/Support-Questions/Does-ExecuteSQL-processor-allow-to-execute-stored-procedure/m-p/158922 https://stackoverflow.com/questions/72348844/get-output-of-stored-procedure-using-putsql-in-nifi If that helps please accept solution. Thanks ... View more

Hi @SAMSAL , that's pretty simple, but works fine for me. Thanks for the advice. Regards Maik ... View more