Member since
03-26-2024
2
Posts
0
Kudos Received
0
Solutions
03-27-2024
09:36 AM
Hi Matt. Thanks for taking the time to help troubleshoot. 1. Is this a standalone single NiFi instance or a NiFi multi-instance cluster setup? standalone single NiFi instance 2. How many partitions on your source NiFi Kafka topic? 4 3. How do you have your MergeContent processor configured? 4. When you say connection quickly fills up, what are the settings on the connection? 5. With your flow running and processing FlowFiles through the dataflow connections, what is the CPU load average. You can find these details from within NiFi's UI from either the cluster UI under global menu in upper right corner or the system diagnostics Ui found in the controller UI also under the global menu. I wasn't able to find this menu to get the average. 6. Do you have a lot of other dataflows also running within this same NiFi? this is the only flow. Here's a screenshot of what the flow looks like while running. You can see the merged queue is full, which happens within a few seconds of the application starting.
... View more
03-26-2024
07:24 AM
I'm new to NiFi. I have a flow that's consuming messages from Kafka and producing them to Redis. The issue I'm experiencing is the MergedContent queue is filling up within seconds, which is slowing down the process of getting the records into Redis. I'm wondering what the recommended approach is to improve things (being new to Redis, I'm aware that I'm most likely missing some obvious things). My flow looks like this:
... View more
Labels:
- Labels:
-
Apache NiFi