Marfill, If backpressure is applied when the total number of flowfiles in a given queue has reached (#nodes * the limit per node) for example if you have a cluster of 3 nodes and the threshold is set to 10,000 then the backpressure will be applied when total # of flow files = 30,000 and so on. Regarding the Control Rate I believe its done per node statistics, for example if you have a control rate that allows 1 flow file per hour and the control rate processor is set part of load balancing on 3 nodes cluster, let say you receive total of 3 files for the first time  one on each node then the 3 will be get processed immediately. ... View more

Hi, Can you post the Expression Language you used? I'm seeing in the error message that its complaining about the "literal" function doesnt exist. Did you use it with the variable name that contains the original date value? If so then remove the literal function and just keep the variable since literal works with constants. So the expression should be something like this:   ${date_variable:replaceAll('(IST)\s',''):toDate('EEE MMM dd HH:mm:ss yyyy'):format('yyyy-MM-dd HH:mm:ss')} ... View more

Hi, If your configurations are similar to mine then I would suggest that you test it on version 1.16.0 to see if its a bug with version  1.16.2 which in this case needs to be reported. Another option for you is to write custom code using ExecuteScript processor that will generate a different flowfile  for each json record since the ExecuteProcess wont have this option and would write everything in the output steam into one flowfile. you can refer to the following post to help you generate multiple flowfiles: https://community.cloudera.com/t5/Support-Questions/Split-one-Nifi-flow-file-into-Multiple-flow-file-based-on/m-p/203387 Another option you can try incase there is a problem with the SplitContent processor is to create every json record in a different line and then try to split them using SplitText processor and set the property "Split Line Count" to 1 ... View more

Hi, I ran into this issue before when dealing with ISO timeformat and it would not allow me to write to the DB. In this case you have to use an updateAttribute to convert this format to the format that DB can accept (yyyy-MM-dd HH:mm:ss), this worked for me: ${literal('Wed Oct 13 15:58:58 IST 2021'):replaceAll('(IST)\s',''):toDate('EEE MMM dd HH:mm:ss yyyy'):format('yyyy-MM-dd HH:mm:ss')} Hope it would help. ... View more

Hi, I'm not aware of any out of the box processor that can help you with that. A suggestion would be to write custom code in an ExecuteScript Processor that gets the filename attribute and stores in a file with the required format in some staging directory, In this processor  you can also decide how many flowfiles you want to process per file and once that file reaches the limit (could be by file size or number of entries) you then move the file to final directory where and GetFile processor is reading from and direct the content of the read file (filenames) to PUTEmail. New file will be created in the staging area to address any new entries after moving the older file to the final directory. ... View more

If you want you can take a a look at the actual source code for existing procssors like GetFile on Github to see if that will help: https://github.com/coco11563/nifi-1.4.0/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetFile.java   ... View more