Skip to main content
brajeshreddy
Gold ⭐️
Jul 25, 2023 2:20 PM UTC
Question

How to use control rate processor based on flow file processing status?

Hi,

 

I have a nifi flow which has 1000's of flow files. Each of the flow file has an impala/hive statement and each of them have different execution times. How do i use control rate processor to trigger next flow file only if the previous flow file has been processed (success/failure of the flow file doesn't matter)?

2 replies

SAMSAL
Platinum ⭐️⭐️⭐️
Jul 25, 2023 6:14 PM UTC

Hi @brajeshreddy ,

I dont think you can achieve this using the ControlRate processor. Instead I recommend the solution mentioned here: https://stackoverflow.com/questions/69893107/nifi-activate-a-flow-file-after-a-previous-flow-file-goes-through-the-downstre

 

If you have a cluster then make sure that the top processor is executing on the Primary node.

 

hope that helps.

Thanks

 

Translate
brajeshreddy
Gold ⭐️
Jul 26, 2023 1:51 PM UTC

@SAMSAL This looks like a plausible solution. Let me try it out. 

Translate