Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Difficulty monitoring data flow. Observed fluctuations in the 5 minute statistics, please help?

Difficulty monitoring data flow. Observed fluctuations in the 5 minute statistics, please help?

New Contributor

I have established a NiFi flow that receives TCP messages at an IOT device (using ListenTCP) and forwards them to a RPG on cloud which eventually pushes it into Cassandra. I have been monitoring the 5 minute statistics to obtain the throughput from the IOT device as well as the rate at which it gets inserted into Cassandra. I have observed that for larger amounts of data, the stats steadily rise till a point and then start fluctuating even though the entire data has not been transferred yet.

1> Can someone please explain this behaviour to me? Is there a something I need to correct/modify?

2> Is there a better alternative to monitor the flow to get the final throughput? I'm interested in obtaining the total time it took for data to reach and leave the RPG and the PutCassandra processors respectively.

Thanks in advance,

Janaki

Don't have an account?
Coming from Hortonworks? Activate your account here