Some examples https://community.cloudera.com/t5/Support-Questions/How-Extract-text-from-a-multiline-flow-and-create-only-one/td-p/104706   https://nathanlabadie.com/recombining-multiline-logs-with/   https://github.com/tspannhw/EverythingApacheNiFi/blob/main/README.md ... View more

Use UpdateRecord (csv reader, json writer) to change date PutDatabaseRecord to save to oracle   https://www.datainmotion.dev/2020/12/simple-change-data-capture-cdc-with-sql.html https://www.datainmotion.dev/2019/10/migrating-apache-flume-flows-to-apache_15.html https://www.datainmotion.dev/2021/01/flank-real-time-transit-information-for.html https://www.datainmotion.dev/2020/12/smart-stocks-with-flank-nifi-kafka.html https://github.com/tspannhw/EverythingApacheNiFi https://www.datainmotion.dev/2021/03/processing-fixed-width-and-complex-files.html https://www.datainmotion.dev/2020/06/no-more-spaghetti-flows.html   ... View more

VMs are not optimal.   Run microservices in Kafka Connect, NiFi Stateless, Flink, Spark or Python in CML, Jupyter Notebooks. SQL Stream builder.   all good options ... View more

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-prometheus-nar/1.11.1/org.apache.nifi.reporting.prometheus.PrometheusReportingTask/   You will need to open a JIRA. You should not run 3 nifis on the same machine unless it's in K8.   NiFi needs a lot of RAM and cores. ... View more

what version of NiFi? you can tweak the log level of each processor in the properties. You can also query the stats with reporting tasks you can also access stats via the rest api https://github.com/tspannhw/EverythingApacheNiFi ... View more

QueryRecordProcessor grabs fields from content You can also add attributes there   You can also add attributes via updaterecord   https://www.datainmotion.dev/2020/12/smart-stocks-with-flank-nifi-kafka.html ... View more

check disk space   http://apache-nifi-users-list.2361937.n4.nabble.com/Clarifications-on-getting-flowfiles-using-FlowFileFilters-td7333.html     ... View more

Unfortunately variables cannot be inside other dynamic parameters.   It is already rendered by then.   What you can do is store this part  select * from tbl_sales where load_date= then updateattribute query ${result:append( "'" +${load_date} +"'")} ... View more

Add more RAM and CPU to your NiFi server Add more NiFi servers to your cluster.   How much is HUGE DATA?    You will need some amount of RAM to process it.   Make sure minimum JVM RAM on each node is a good size for big workloads.   32G at least. ... View more