I am using NiFi 1.11.4  Currently log of "Input port" is available in the log file but the count of the flow file is not present. Do you think log level will help there?  ... View more

QueryRecordProcessor grabs fields from content You can also add attributes there   You can also add attributes via updaterecord   https://www.datainmotion.dev/2020/12/smart-stocks-with-flank-nifi-kafka.html ... View more

check disk space   http://apache-nifi-users-list.2361937.n4.nabble.com/Clarifications-on-getting-flowfiles-using-FlowFileFilters-td7333.html     ... View more

Thanks. We have decided to introduce Priority attribute & use the corresponding prioritizer for the connection. ... View more

Add more RAM and CPU to your NiFi server Add more NiFi servers to your cluster.   How much is HUGE DATA?    You will need some amount of RAM to process it.   Make sure minimum JVM RAM on each node is a good size for big workloads.   32G at least. ... View more

Put both tables in Kafka topics and have SQL Stream Builder joing them with a simple SQL Join.   or https://community.cloudera.com/t5/Support-Questions/Nifi-how-to-sql-join-two-flowfiles/td-p/298227   http://apache-nifi-users-list.2361937.n4.nabble.com/Joining-two-or-more-flow-files-and-merging-the-content-td10543.html   https://medium.com/@surajnagendra/merge-csv-files-apache-nifi-21ba44e1b719   ... View more

@Masi  The exception does not appear related to Load Balance connections in NiFi.  LB Connections utilize NiFi S2S in the background which does not use MySQL.  Matt ... View more

Move all events to Kafka ... View more

What are the column names in your table? Assuming "carId" and "carType", you can use JoltTransformJson or JoltTransformRecord with the following spec:   [ { "operation": "shift", "spec": { "*": { "$": "carId", "@": "carType" } } }, { "operation": "shift", "spec": { "carId": { "*": { "@": "[&0].carId" } }, "carType": { "*": { "@": "[&0].carType" } } } } ]     ... View more

NiFi versions are tied to Hive versions so you need to compatible one.   Check with your Cloudera team to ge tthe correct version.   Using PutHive3Streaming will be faster.   So is just PutOrc, PutParquet or PutHDFS. ... View more